Hello community, here is the log from the commit of package python-waitress for openSUSE:Factory checked in at 2014-12-16 14:48:42 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/python-waitress (Old) and /work/SRC/openSUSE:Factory/.python-waitress.new (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "python-waitress" Changes: -------- --- /work/SRC/openSUSE:Factory/python-waitress/python-waitress.changes 2013-09-13 14:46:22.000000000 +0200 +++ /work/SRC/openSUSE:Factory/.python-waitress.new/python-waitress.changes 2014-12-16 14:48:38.000000000 +0100 @@ -1,0 +2,59 @@ +Fri Dec 12 22:08:09 UTC 2014 - tbecht...@suse.com + +- update to 0.8.9: + - Fix tests under Windows. NB: to run tests under Windows, you cannot run + "setup.py test" or "setup.py nosetests". Instead you must run ``python.exe + -c "import nose; nose.main()"``. If you try to run the tests using the + normal method under Windows, each subprocess created by the test suite will + attempt to run the test suite again. See + https://github.com/nose-devs/nose/issues/407 for more information. + - Give the WSGI app_iter generated when ``wsgi.file_wrapper`` is used + (ReadOnlyFileBasedBuffer) a ``close`` method. Do not call ``close`` on an + instance of such a class when it's used as a WSGI app_iter, however. This is + part of a fix which prevents a leakage of file descriptors; the other part of + the fix was in WebOb + (https://github.com/Pylons/webob/commit/951a41ce57bd853947f842028bccb500bd5237da). + - Allow trusted proxies to override ``wsgi.url_scheme`` via a request header, + ``X_FORWARDED_PROTO``. Allows proxies which serve mixed HTTP / HTTPS + requests to control signal which are served as HTTPS. See + https://github.com/Pylons/waitress/pull/42. + - Fix some cases where the creation of extremely large output buffers (greater + than 2GB, suspected to be buffers added via ``wsgi.file_wrapper``) might + cause an OverflowError on Python 2. See + https://github.com/Pylons/waitress/issues/47. + - When the ``url_prefix`` adjustment starts with more than one slash, all + slashes except one will be stripped from its beginning. This differs from + older behavior where more than one leading slash would be preserved in + ``url_prefix``. + - If a client somehow manages to send an empty path, we no longer convert the + empty path to a single slash in ``PATH_INFO``. Instead, the path remains + empty. According to RFC 2616 section "5.1.2 Request-URI", the scenario of a + client sending an empty path is actually not possible because the request URI + portion cannot be empty. + - If the ``url_prefix`` adjustment matches the request path exactly, we now + compute ``SCRIPT_NAME`` and ``PATH_INFO`` properly. Previously, if the + ``url_prefix`` was ``/foo`` and the path received from a client was ``/foo``, + we would set *both* ``SCRIPT_NAME`` and ``PATH_INFO`` to ``/foo``. This was + incorrect. Now in such a case we set ``PATH_INFO`` to the empty string and + we set ``SCRIPT_NAME`` to ``/foo``. Note that the change we made has no + effect on paths that do not match the ``url_prefix`` exactly (such as + ``/foo/bar``); these continue to operate as they did. See + https://github.com/Pylons/waitress/issues/46 + - Preserve header ordering of headers with the same name as per RFC 2616. See + https://github.com/Pylons/waitress/pull/44 + - When waitress receives a ``Transfer-Encoding: chunked`` request, we no longer + send the ``TRANSFER_ENCODING`` nor the ``HTTP_TRANSFER_ENCODING`` value to + the application in the environment. Instead, we pop this header. Since we + cope with chunked requests by buffering the data in the server, we also know + when a chunked request has ended, and therefore we know the content length. + We set the content-length header in the environment, such that applications + effectively never know the original request was a T-E: chunked request; it + will appear to them as if the request is a non-chunked request with an + accurate content-length. + - Cope with the fact that the ``Transfer-Encoding`` value is case-insensitive. + - When the ``--unix-socket-perms`` option was used as an argument to + ``waitress-serve``, a ``TypeError`` would be raised. See + https://github.com/Pylons/waitress/issues/50. +- Enable testsuite during build + +------------------------------------------------------------------- Old: ---- waitress-0.8.7.tar.gz New: ---- waitress-0.8.9.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ python-waitress.spec ++++++ --- /var/tmp/diff_new_pack.eCvnni/_old 2014-12-16 14:48:39.000000000 +0100 +++ /var/tmp/diff_new_pack.eCvnni/_new 2014-12-16 14:48:39.000000000 +0100 @@ -1,7 +1,7 @@ # # spec file for package python-waitress # -# Copyright (c) 2013 SUSE LINUX Products GmbH, Nuernberg, Germany. +# Copyright (c) 2014 SUSE LINUX Products GmbH, Nuernberg, Germany. # # All modifications and additions to the file contributed by third parties # remain the property of their copyright owners, unless otherwise agreed @@ -17,7 +17,7 @@ Name: python-waitress -Version: 0.8.7 +Version: 0.8.9 Release: 0 Summary: Waitress WSGI server License: ZPL-2.1 @@ -27,7 +27,7 @@ BuildRequires: python-devel BuildRequires: python-setuptools # Test requirements: -#BuildRequires: python-nose +BuildRequires: python-nose # Documentation requirements: #BuildRequires: python-Sphinx #BuildRequires: python-docutils @@ -58,8 +58,8 @@ %install python setup.py install --prefix=%{_prefix} --root=%{buildroot} -#%%check -#python setup.py test +%check +python setup.py test %files %defattr(-,root,root,-) ++++++ waitress-0.8.7.tar.gz -> waitress-0.8.9.tar.gz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/waitress-0.8.7/CHANGES.txt new/waitress-0.8.9/CHANGES.txt --- old/waitress-0.8.7/CHANGES.txt 2013-08-29 17:54:01.000000000 +0200 +++ new/waitress-0.8.9/CHANGES.txt 2014-05-16 23:47:23.000000000 +0200 @@ -1,3 +1,73 @@ +0.8.9 (2014-05-16) +------------------ + +- Fix tests under Windows. NB: to run tests under Windows, you cannot run + "setup.py test" or "setup.py nosetests". Instead you must run ``python.exe + -c "import nose; nose.main()"``. If you try to run the tests using the + normal method under Windows, each subprocess created by the test suite will + attempt to run the test suite again. See + https://github.com/nose-devs/nose/issues/407 for more information. + +- Give the WSGI app_iter generated when ``wsgi.file_wrapper`` is used + (ReadOnlyFileBasedBuffer) a ``close`` method. Do not call ``close`` on an + instance of such a class when it's used as a WSGI app_iter, however. This is + part of a fix which prevents a leakage of file descriptors; the other part of + the fix was in WebOb + (https://github.com/Pylons/webob/commit/951a41ce57bd853947f842028bccb500bd5237da). + +- Allow trusted proxies to override ``wsgi.url_scheme`` via a request header, + ``X_FORWARDED_PROTO``. Allows proxies which serve mixed HTTP / HTTPS + requests to control signal which are served as HTTPS. See + https://github.com/Pylons/waitress/pull/42. + +0.8.8 (2013-11-30) +------------------ + +- Fix some cases where the creation of extremely large output buffers (greater + than 2GB, suspected to be buffers added via ``wsgi.file_wrapper``) might + cause an OverflowError on Python 2. See + https://github.com/Pylons/waitress/issues/47. + +- When the ``url_prefix`` adjustment starts with more than one slash, all + slashes except one will be stripped from its beginning. This differs from + older behavior where more than one leading slash would be preserved in + ``url_prefix``. + +- If a client somehow manages to send an empty path, we no longer convert the + empty path to a single slash in ``PATH_INFO``. Instead, the path remains + empty. According to RFC 2616 section "5.1.2 Request-URI", the scenario of a + client sending an empty path is actually not possible because the request URI + portion cannot be empty. + +- If the ``url_prefix`` adjustment matches the request path exactly, we now + compute ``SCRIPT_NAME`` and ``PATH_INFO`` properly. Previously, if the + ``url_prefix`` was ``/foo`` and the path received from a client was ``/foo``, + we would set *both* ``SCRIPT_NAME`` and ``PATH_INFO`` to ``/foo``. This was + incorrect. Now in such a case we set ``PATH_INFO`` to the empty string and + we set ``SCRIPT_NAME`` to ``/foo``. Note that the change we made has no + effect on paths that do not match the ``url_prefix`` exactly (such as + ``/foo/bar``); these continue to operate as they did. See + https://github.com/Pylons/waitress/issues/46 + +- Preserve header ordering of headers with the same name as per RFC 2616. See + https://github.com/Pylons/waitress/pull/44 + +- When waitress receives a ``Transfer-Encoding: chunked`` request, we no longer + send the ``TRANSFER_ENCODING`` nor the ``HTTP_TRANSFER_ENCODING`` value to + the application in the environment. Instead, we pop this header. Since we + cope with chunked requests by buffering the data in the server, we also know + when a chunked request has ended, and therefore we know the content length. + We set the content-length header in the environment, such that applications + effectively never know the original request was a T-E: chunked request; it + will appear to them as if the request is a non-chunked request with an + accurate content-length. + +- Cope with the fact that the ``Transfer-Encoding`` value is case-insensitive. + +- When the ``--unix-socket-perms`` option was used as an argument to + ``waitress-serve``, a ``TypeError`` would be raised. See + https://github.com/Pylons/waitress/issues/50. + 0.8.7 (2013-08-29) ------------------ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/waitress-0.8.7/PKG-INFO new/waitress-0.8.9/PKG-INFO --- old/waitress-0.8.7/PKG-INFO 2013-08-29 17:57:01.000000000 +0200 +++ new/waitress-0.8.9/PKG-INFO 2014-05-16 23:48:21.000000000 +0200 @@ -1,6 +1,6 @@ Metadata-Version: 1.1 Name: waitress -Version: 0.8.7 +Version: 0.8.9 Summary: Waitress WSGI server Home-page: https://github.com/Pylons/waitress Author: Chris McDonough @@ -16,6 +16,76 @@ http://docs.pylonsproject.org/projects/waitress/en/latest/ . + 0.8.9 (2014-05-16) + ------------------ + + - Fix tests under Windows. NB: to run tests under Windows, you cannot run + "setup.py test" or "setup.py nosetests". Instead you must run ``python.exe + -c "import nose; nose.main()"``. If you try to run the tests using the + normal method under Windows, each subprocess created by the test suite will + attempt to run the test suite again. See + https://github.com/nose-devs/nose/issues/407 for more information. + + - Give the WSGI app_iter generated when ``wsgi.file_wrapper`` is used + (ReadOnlyFileBasedBuffer) a ``close`` method. Do not call ``close`` on an + instance of such a class when it's used as a WSGI app_iter, however. This is + part of a fix which prevents a leakage of file descriptors; the other part of + the fix was in WebOb + (https://github.com/Pylons/webob/commit/951a41ce57bd853947f842028bccb500bd5237da). + + - Allow trusted proxies to override ``wsgi.url_scheme`` via a request header, + ``X_FORWARDED_PROTO``. Allows proxies which serve mixed HTTP / HTTPS + requests to control signal which are served as HTTPS. See + https://github.com/Pylons/waitress/pull/42. + + 0.8.8 (2013-11-30) + ------------------ + + - Fix some cases where the creation of extremely large output buffers (greater + than 2GB, suspected to be buffers added via ``wsgi.file_wrapper``) might + cause an OverflowError on Python 2. See + https://github.com/Pylons/waitress/issues/47. + + - When the ``url_prefix`` adjustment starts with more than one slash, all + slashes except one will be stripped from its beginning. This differs from + older behavior where more than one leading slash would be preserved in + ``url_prefix``. + + - If a client somehow manages to send an empty path, we no longer convert the + empty path to a single slash in ``PATH_INFO``. Instead, the path remains + empty. According to RFC 2616 section "5.1.2 Request-URI", the scenario of a + client sending an empty path is actually not possible because the request URI + portion cannot be empty. + + - If the ``url_prefix`` adjustment matches the request path exactly, we now + compute ``SCRIPT_NAME`` and ``PATH_INFO`` properly. Previously, if the + ``url_prefix`` was ``/foo`` and the path received from a client was ``/foo``, + we would set *both* ``SCRIPT_NAME`` and ``PATH_INFO`` to ``/foo``. This was + incorrect. Now in such a case we set ``PATH_INFO`` to the empty string and + we set ``SCRIPT_NAME`` to ``/foo``. Note that the change we made has no + effect on paths that do not match the ``url_prefix`` exactly (such as + ``/foo/bar``); these continue to operate as they did. See + https://github.com/Pylons/waitress/issues/46 + + - Preserve header ordering of headers with the same name as per RFC 2616. See + https://github.com/Pylons/waitress/pull/44 + + - When waitress receives a ``Transfer-Encoding: chunked`` request, we no longer + send the ``TRANSFER_ENCODING`` nor the ``HTTP_TRANSFER_ENCODING`` value to + the application in the environment. Instead, we pop this header. Since we + cope with chunked requests by buffering the data in the server, we also know + when a chunked request has ended, and therefore we know the content length. + We set the content-length header in the environment, such that applications + effectively never know the original request was a T-E: chunked request; it + will appear to them as if the request is a non-chunked request with an + accurate content-length. + + - Cope with the fact that the ``Transfer-Encoding`` value is case-insensitive. + + - When the ``--unix-socket-perms`` option was used as an argument to + ``waitress-serve``, a ``TypeError`` would be raised. See + https://github.com/Pylons/waitress/issues/50. + 0.8.7 (2013-08-29) ------------------ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/waitress-0.8.7/docs/arguments.rst new/waitress-0.8.9/docs/arguments.rst --- old/waitress-0.8.7/docs/arguments.rst 2013-08-12 06:42:35.000000000 +0200 +++ new/waitress-0.8.9/docs/arguments.rst 2014-05-16 23:14:59.000000000 +0200 @@ -28,8 +28,14 @@ number of threads used to process application logic (integer), default ``4`` +trusted_proxy + IP address of a client allowed to override ``url_scheme`` via the + ``X_FORWARDED_PROTO`` header. + url_scheme - default ``wsgi.url_scheme`` value (string), default ``http`` + default ``wsgi.url_scheme`` value (string), default ``http``; can be + overridden per-request by the value of the ``X_FORWARDED_PROTO`` header, + but only if the client address matches ``trusted_proxy``. ident server identity (string) used in "Server:" header in responses, default diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/waitress-0.8.7/docs/conf.py new/waitress-0.8.9/docs/conf.py --- old/waitress-0.8.7/docs/conf.py 2013-08-29 17:54:29.000000000 +0200 +++ new/waitress-0.8.9/docs/conf.py 2014-05-16 23:39:35.000000000 +0200 @@ -18,6 +18,7 @@ #sys.path.append(os.path.abspath('some/directory')) import sys, os +import pkg_resources # Add and use Pylons theme if 'sphinx-build' in ' '.join(sys.argv): # protect against dumb importers @@ -65,7 +66,7 @@ # other places throughout the built documents. # # The short X.Y version. -version = '0.8.7' +version = pkg_resources.get_distribution('waitress').version # The full version, including alpha/beta/rc tags. release = version diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/waitress-0.8.7/docs/differences.rst new/waitress-0.8.9/docs/differences.rst --- old/waitress-0.8.7/docs/differences.rst 2012-01-16 10:02:05.000000000 +0100 +++ new/waitress-0.8.9/docs/differences.rst 2014-05-16 23:14:59.000000000 +0200 @@ -13,6 +13,9 @@ - Calls "close()" on the app_iter object returned by the WSGI application. +- Allows trusted proxies to override ``wsgi.url_scheme`` for particular + requests by supplying the ``X_FORWARDED_PROTO`` header. + - Supports an explicit ``wsgi.url_scheme`` parameter for ease of deployment behind SSL proxies. diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/waitress-0.8.7/docs/index.rst new/waitress-0.8.9/docs/index.rst --- old/waitress-0.8.7/docs/index.rst 2013-08-12 06:58:04.000000000 +0200 +++ new/waitress-0.8.9/docs/index.rst 2014-05-16 23:14:59.000000000 +0200 @@ -57,7 +57,7 @@ can be used in development and in situations where the likes of :term:`PasteDeploy` is not necessary:: - waitress-serve --port-8041 myapp:wsgifunc + waitress-serve --port=8041 myapp:wsgifunc For more information on this, see :ref:`runner`. @@ -119,12 +119,18 @@ used behind a reverse proxy served by Waitress might inappropriately be ``http://foo`` rather than ``https://foo``. To fix this, you'll want to change the ``wsgi.url_scheme`` in the WSGI environment before it reaches your -application. You can do this in one of two ways: +application. You can do this in one of three ways: 1. You can pass a ``url_scheme`` configuration variable to the ``waitress.serve`` function. -2. You can use Paste's ``PrefixMiddleware`` in conjunction with +2. You can configure the proxy reverse server to pass a header, + ``X_FORWARDED_PROTO``, whose value will be set for that request as + the ``wsgi.url_scheme`` environment value. Note that you must also + conigure ``waitress.serve`` by passing the IP address of that proxy + as its ``trusted_proxy``. + +3. You can use Paste's ``PrefixMiddleware`` in conjunction with configuration settings on the reverse proxy server. Using ``url_scheme`` to set ``wsgi.url_scheme`` @@ -138,6 +144,26 @@ This works if all URLs generated by your application should use the ``https`` scheme. +Passing the ``X_FORWARDED_PROTO`` header to set ``wsgi.url_scheme`` +------------------------------------------------------------------- + +If your proxy accepts both HTTP and HTTPS URLs, and you want your application +to generate the appropriate url based on the incoming scheme, also set up +your proxy to send a ``X-Forwarded-Proto`` with the original URL scheme along +with each proxied request. For example, when using Nginx:: + + proxy_set_header X-Forwarded-Proto $scheme; + +or via Apache:: + + RequestHeader set X-Forwarded-Proto https + +.. note:: + + You must also configure the Waitress server's ``trusted_proxy`` to + contain the IP address of the proxy in order for this header to override + the default URL scheme. + Using ``url_prefix`` to influence ``SCRIPT_NAME`` and ``PATH_INFO`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/waitress-0.8.7/docs/runner.rst new/waitress-0.8.9/docs/runner.rst --- old/waitress-0.8.7/docs/runner.rst 2013-08-12 06:58:29.000000000 +0200 +++ new/waitress-0.8.9/docs/runner.rst 2014-05-16 23:14:59.000000000 +0200 @@ -121,7 +121,7 @@ Number of bytes to request when calling ``socket.recv()``. Default is 8192. -``--send-bytes=INT``` +``--send-bytes=INT`` Number of bytes to send to socket.send(). Default is 18000. Multiples of 9000 should avoid partly-filled TCP packets. diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/waitress-0.8.7/setup.py new/waitress-0.8.9/setup.py --- old/waitress-0.8.7/setup.py 2013-08-29 17:54:21.000000000 +0200 +++ new/waitress-0.8.9/setup.py 2014-05-16 23:47:31.000000000 +0200 @@ -37,7 +37,7 @@ setup( name='waitress', - version='0.8.7', + version='0.8.9', author='Zope Foundation and Contributors', author_email='zope-...@zope.org', maintainer="Chris McDonough", diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/waitress-0.8.7/waitress/adjustments.py new/waitress-0.8.9/waitress/adjustments.py --- old/waitress-0.8.7/waitress/adjustments.py 2013-08-12 06:44:02.000000000 +0200 +++ new/waitress-0.8.9/waitress/adjustments.py 2014-05-16 23:14:59.000000000 +0200 @@ -36,8 +36,13 @@ """Convert the given octal string to an actual number.""" return int(s, 8) -def slash_suffix_stripped_str(s): - return s.rstrip('/') +def slash_fixed_str(s): + s = s.strip() + if s: + # always have a leading slash, replace any number of leading slashes + # with a single slash, and strip any trailing slashes + s = '/' + s.lstrip('/').rstrip('/') + return s class Adjustments(object): """This class contains tunable parameters. @@ -47,8 +52,9 @@ ('host', str), ('port', int), ('threads', int), + ('trusted_proxy', str), ('url_scheme', str), - ('url_prefix', slash_suffix_stripped_str), + ('url_prefix', slash_fixed_str), ('backlog', int), ('recv_bytes', int), ('send_bytes', int), @@ -79,6 +85,9 @@ # mumber of threads available for tasks threads = 4 + # Host allowed to overrid ``wsgi.url_scheme`` via header + trusted_proxy = None + # default ``wsgi.url_scheme`` value url_scheme = 'http' @@ -176,7 +185,10 @@ @classmethod def parse_args(cls, argv): - """Parse command line arguments. + """Pre-parse command line arguments for input into __init__. Note that + this does not cast values into adjustment types, it just creates a + dictionary suitable for passing into __init__, where __init__ does the + casting. """ long_opts = ['help', 'call'] for opt, cast in cls._params: @@ -196,9 +208,11 @@ param = opt.lstrip('-').replace('-', '_') if param.startswith('no_'): param = param[3:] - kw[param] = False - elif param in ('help', 'call') or cls._param_map[param] is asbool: + kw[param] = 'false' + elif param in ('help', 'call'): kw[param] = True + elif cls._param_map[param] is asbool: + kw[param] = 'true' else: - kw[param] = cls._param_map[param](value) + kw[param] = value return kw, args diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/waitress-0.8.7/waitress/buffers.py new/waitress-0.8.9/waitress/buffers.py --- old/waitress-0.8.7/waitress/buffers.py 2013-05-24 01:02:12.000000000 +0200 +++ new/waitress-0.8.9/waitress/buffers.py 2014-05-16 23:14:59.000000000 +0200 @@ -103,9 +103,7 @@ def getfile(self): return self.file - def _close(self): - # named _close because ReadOnlyFileBasedBuffer is used as - # wsgi file.wrapper, and its protocol reserves "close" + def close(self): if hasattr(self.file, 'close'): self.file.close() self.remain = 0 @@ -149,9 +147,6 @@ self.remain = fsize else: self.remain = min(fsize, size) - elif hasattr(self.file, 'close'): - # called by task if self.filelike has no seek/tell - self.close = self.file.close return self.remain def get(self, numbytes=-1, skip=False): @@ -186,8 +181,8 @@ """ This buffer implementation has four stages: - No data - - String-based buffer - - StringIO-based buffer + - Bytes-based buffer + - BytesIO-based buffer - Temporary file storage The first two stages are fastest for simple transfers. """ @@ -203,11 +198,15 @@ def __len__(self): buf = self.buf if buf is not None: + # use buf.__len__ rather than len(buf) FBO of not getting + # OverflowError on Python 2 return buf.__len__() else: return self.strbuf.__len__() def __nonzero__(self): + # use self.__len__ rather than len(self) FBO of not getting + # OverflowError on Python 2 return self.__len__() > 0 __bool__ = __nonzero__ # py3 @@ -241,7 +240,9 @@ return buf = self._create_buffer() buf.append(s) - sz = len(buf) + # use buf.__len__ rather than len(buf) FBO of not getting + # OverflowError on Python 2 + sz = buf.__len__() if not self.overflowed: if sz >= self.overflow: self._set_large_buffer() @@ -278,7 +279,9 @@ return buf.prune() if self.overflowed: - sz = len(buf) + # use buf.__len__ rather than len(buf) FBO of not getting + # OverflowError on Python 2 + sz = buf.__len__() if sz < self.overflow: # Revert to a faster buffer. self._set_small_buffer() @@ -289,7 +292,7 @@ buf = self._create_buffer() return buf.getfile() - def _close(self): + def close(self): buf = self.buf if buf is not None: - buf._close() + buf.close() diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/waitress-0.8.7/waitress/channel.py new/waitress-0.8.9/waitress/channel.py --- old/waitress-0.8.7/waitress/channel.py 2013-08-29 17:53:38.000000000 +0200 +++ new/waitress-0.8.9/waitress/channel.py 2014-05-16 23:14:59.000000000 +0200 @@ -89,7 +89,10 @@ return False def total_outbufs_len(self): - return sum([len(b) for b in self.outbufs]) # genexpr == more funccalls + # genexpr == more funccalls + # use b.__len__ rather than len(b) FBO of not getting OverflowError + # on Python 2 + return sum([b.__len__() for b in self.outbufs]) def writable(self): # if there's data in the out buffer or we've been instructed to close @@ -233,13 +236,15 @@ while True: outbuf = self.outbufs[0] - outbuflen = len(outbuf) + # use outbuf.__len__ rather than len(outbuf) FBO of not getting + # OverflowError on Python 2 + outbuflen = outbuf.__len__() if outbuflen <= 0: # self.outbufs[-1] must always be a writable outbuf if len(self.outbufs) > 1: toclose = self.outbufs.pop(0) try: - toclose._close() + toclose.close() except: self.logger.exception( 'Unexpected error when closing an outbuf') @@ -270,7 +275,7 @@ def handle_close(self): for outbuf in self.outbufs: try: - outbuf._close() + outbuf.close() except: self.logger.exception( 'Unknown exception while trying to close outbuf') @@ -360,11 +365,11 @@ if task.close_on_finish: self.close_when_flushed = True for request in self.requests: - request._close() + request.close() self.requests = [] else: request = self.requests.pop(0) - request._close() + request.close() self.force_flush = True self.server.pull_trigger() diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/waitress-0.8.7/waitress/compat.py new/waitress-0.8.9/waitress/compat.py --- old/waitress-0.8.7/waitress/compat.py 2013-05-24 01:02:12.000000000 +0200 +++ new/waitress-0.8.9/waitress/compat.py 2014-05-16 23:14:59.000000000 +0200 @@ -109,3 +109,8 @@ import httplib except ImportError: # pragma: no cover from http import client as httplib + +try: + MAXINT = sys.maxint +except AttributeError: # pragma: no cover + MAXINT = sys.maxsize diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/waitress-0.8.7/waitress/parser.py new/waitress-0.8.9/waitress/parser.py --- old/waitress-0.8.7/waitress/parser.py 2013-08-29 17:53:38.000000000 +0200 +++ new/waitress-0.8.9/waitress/parser.py 2014-05-16 23:14:59.000000000 +0200 @@ -148,7 +148,16 @@ self.error = br.error self.completed = True elif br.completed: + # The request (with the body) is ready to use. self.completed = True + if self.chunked: + # We've converted the chunked transfer encoding request + # body into a normal request body, so we know its content + # length; set the header here. We already popped the + # TRANSFER_ENCODING header in parse_header, so this will + # appear to the client to be an entirely non-chunked HTTP + # request with a valid content-length. + self.headers['CONTENT_LENGTH'] = str(br.__len__()) return consumed def parse_header(self, header_plus): @@ -203,8 +212,12 @@ self.connection_close = True if version == '1.1': - te = headers.get('TRANSFER_ENCODING', '') - if te == 'chunked': + # since the server buffers data from chunked transfers and clients + # never need to deal with chunked requests, downstream clients + # should not see the HTTP_TRANSFER_ENCODING header; we pop it + # here + te = headers.pop('TRANSFER_ENCODING', '') + if te.lower() == 'chunked': self.chunked = True buf = OverflowableBuffer(self.adj.inbuf_overflow) self.body_rcv = ChunkedReceiver(buf) @@ -230,10 +243,10 @@ else: return BytesIO() - def _close(self): + def close(self): body_rcv = self.body_rcv if body_rcv is not None: - body_rcv.getbuf()._close() + body_rcv.getbuf().close() def split_uri(uri): # urlsplit handles byte input by returning bytes on py3, so diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/waitress-0.8.7/waitress/receiver.py new/waitress-0.8.9/waitress/receiver.py --- old/waitress-0.8.7/waitress/receiver.py 2013-05-24 01:02:12.000000000 +0200 +++ new/waitress-0.8.9/waitress/receiver.py 2014-05-16 23:14:59.000000000 +0200 @@ -28,6 +28,9 @@ self.remain = cl self.buf = buf + def __len__(self): + return self.buf.__len__() + def received(self, data): 'See IStreamConsumer' rm = self.remain @@ -66,6 +69,9 @@ def __init__(self, buf): self.buf = buf + def __len__(self): + return self.buf.__len__() + def received(self, s): # Returns the number of bytes consumed. if self.completed: diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/waitress-0.8.7/waitress/task.py new/waitress-0.8.9/waitress/task.py --- old/waitress-0.8.7/waitress/task.py 2013-08-29 17:53:38.000000000 +0200 +++ new/waitress-0.8.9/waitress/task.py 2014-05-16 23:14:59.000000000 +0200 @@ -255,7 +255,11 @@ response_headers.append(('Date', build_http_date(self.start_time))) first_line = 'HTTP/%s %s' % (self.version, self.status) - next_lines = ['%s: %s' % hv for hv in sorted(self.response_headers)] + # NB: sorting headers needs to preserve same-named-header order + # as per RFC 2616 section 4.2; thus the key=lambda x: x[0] here; + # rely on stable sort to keep relative position of same-named headers + next_lines = ['%s: %s' % hv for hv in sorted( + self.response_headers, key=lambda x: x[0])] lines = [first_line] + next_lines res = '%s\r\n\r\n' % '\r\n'.join(lines) return tobytes(res) @@ -387,19 +391,25 @@ # Call the application to handle the request and write a response app_iter = self.channel.server.application(env, start_response) - try: - if app_iter.__class__ is ReadOnlyFileBasedBuffer: - cl = self.content_length - size = app_iter.prepare(cl) - if size: - if cl != size: - if cl is not None: - self.remove_content_length_header() - self.content_length = size - self.write(b'') # generate headers - self.channel.write_soon(app_iter) - return + if app_iter.__class__ is ReadOnlyFileBasedBuffer: + # NB: do not put this inside the below try: finally: which closes + # the app_iter; we need to defer closing the underlying file. It's + # intention that we don't want to call ``close`` here if the + # app_iter is a ROFBB; the buffer (and therefore the file) will + # eventually be closed within channel.py's _flush_some or + # handle_close instead. + cl = self.content_length + size = app_iter.prepare(cl) + if size: + if cl != size: + if cl is not None: + self.remove_content_length_header() + self.content_length = size + self.write(b'') # generate headers + self.channel.write_soon(app_iter) + return + try: first_chunk_len = None for chunk in app_iter: if first_chunk_len is None: @@ -447,13 +457,28 @@ path = request.path channel = self.channel server = channel.server + url_prefix = server.adj.url_prefix - path = path.lstrip('/') - - url_prefix_with_slash = server.adj.url_prefix.lstrip('/') + '/' - - if url_prefix_with_slash and path.startswith(url_prefix_with_slash): - path = path[len(url_prefix_with_slash):] + if path.startswith('/'): + # strip extra slashes at the beginning of a path that starts + # with any number of slashes + path = '/' + path.lstrip('/') + + if url_prefix: + # NB: url_prefix is guaranteed by the configuration machinery to + # be either the empty string or a string that starts with a single + # slash and ends without any slashes + if path == url_prefix: + # if the path is the same as the url prefix, the SCRIPT_NAME + # should be the url_prefix and PATH_INFO should be empty + path = '' + else: + # if the path starts with the url prefix plus a slash, + # the SCRIPT_NAME should be the url_prefix and PATH_INFO should + # the value of path from the slash until its end + url_prefix_with_trailing_slash = url_prefix + '/' + if path.startswith(url_prefix_with_trailing_slash): + path = path[len(url_prefix):] environ = {} environ['REQUEST_METHOD'] = request.command.upper() @@ -461,12 +486,20 @@ environ['SERVER_NAME'] = server.server_name environ['SERVER_SOFTWARE'] = server.adj.ident environ['SERVER_PROTOCOL'] = 'HTTP/%s' % self.version - environ['SCRIPT_NAME'] = server.adj.url_prefix - environ['PATH_INFO'] = '/' + path + environ['SCRIPT_NAME'] = url_prefix + environ['PATH_INFO'] = path environ['QUERY_STRING'] = request.query - environ['REMOTE_ADDR'] = channel.addr[0] + host = environ['REMOTE_ADDR'] = channel.addr[0] - for key, value in request.headers.items(): + headers = dict(request.headers) + if host == server.adj.trusted_proxy: + wsgi_url_scheme = headers.pop('X_FORWARDED_PROTO', + request.url_scheme) + else: + wsgi_url_scheme = request.url_scheme + if wsgi_url_scheme not in ('http', 'https'): + raise ValueError('Invalid X_FORWARDED_PROTO value') + for key, value in headers.items(): value = value.strip() mykey = rename_headers.get(key, None) if mykey is None: @@ -476,7 +509,7 @@ # the following environment variables are required by the WSGI spec environ['wsgi.version'] = (1, 0) - environ['wsgi.url_scheme'] = request.url_scheme + environ['wsgi.url_scheme'] = wsgi_url_scheme environ['wsgi.errors'] = sys.stderr # apps should use the logging module environ['wsgi.multithread'] = True environ['wsgi.multiprocess'] = False diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/waitress-0.8.7/waitress/tests/fixtureapps/getline.py new/waitress-0.8.9/waitress/tests/fixtureapps/getline.py --- old/waitress-0.8.7/waitress/tests/fixtureapps/getline.py 2013-05-24 01:02:12.000000000 +0200 +++ new/waitress-0.8.9/waitress/tests/fixtureapps/getline.py 2014-05-16 23:15:02.000000000 +0200 @@ -2,13 +2,16 @@ if __name__ == '__main__': try: - from urllib.request import urlopen + from urllib.request import urlopen, URLError except ImportError: - from urllib2 import urlopen + from urllib2 import urlopen, URLError url = sys.argv[1] headers = {'Content-Type': 'text/plain; charset=utf-8'} - resp = urlopen(url) - line = resp.readline().decode('ascii') # py3 + try: + resp = urlopen(url) + line = resp.readline().decode('ascii') # py3 + except URLError: + line = 'failed to read %s' % url sys.stdout.write(line) sys.stdout.flush() diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/waitress-0.8.7/waitress/tests/test_adjustments.py new/waitress-0.8.9/waitress/tests/test_adjustments.py --- old/waitress-0.8.7/waitress/tests/test_adjustments.py 2013-08-12 06:45:42.000000000 +0200 +++ new/waitress-0.8.9/waitress/tests/test_adjustments.py 2014-05-16 23:14:59.000000000 +0200 @@ -51,18 +51,34 @@ def test_goodvars(self): inst = self._makeOne( - host='host', port='8080', threads='5', - url_scheme='https', backlog='20', recv_bytes='200', - send_bytes='300', outbuf_overflow='400', inbuf_overflow='500', - connection_limit='1000', cleanup_interval='1100', - channel_timeout='1200', log_socket_errors='true', - max_request_header_size='1300', max_request_body_size='1400', - expose_tracebacks='true', ident='abc', asyncore_loop_timeout='5', - asyncore_use_poll=True, unix_socket='/tmp/waitress.sock', - unix_socket_perms='777', url_prefix='/foo') + host='host', + port='8080', + threads='5', + trusted_proxy='192.168.1.1', + url_scheme='https', + backlog='20', + recv_bytes='200', + send_bytes='300', + outbuf_overflow='400', + inbuf_overflow='500', + connection_limit='1000', + cleanup_interval='1100', + channel_timeout='1200', + log_socket_errors='true', + max_request_header_size='1300', + max_request_body_size='1400', + expose_tracebacks='true', + ident='abc', + asyncore_loop_timeout='5', + asyncore_use_poll=True, + unix_socket='/tmp/waitress.sock', + unix_socket_perms='777', + url_prefix='///foo/', + ) self.assertEqual(inst.host, 'host') self.assertEqual(inst.port, 8080) self.assertEqual(inst.threads, 5) + self.assertEqual(inst.trusted_proxy, '192.168.1.1') self.assertEqual(inst.url_scheme, 'https') self.assertEqual(inst.backlog, 20) self.assertEqual(inst.recv_bytes, 200) @@ -114,12 +130,12 @@ def test_positive_boolean(self): opts, args = self.parse(['--expose-tracebacks']) - self.assertDictContainsSubset({'expose_tracebacks': True}, opts) + self.assertDictContainsSubset({'expose_tracebacks': 'true'}, opts) self.assertSequenceEqual(args, []) def test_negative_boolean(self): opts, args = self.parse(['--no-expose-tracebacks']) - self.assertDictContainsSubset({'expose_tracebacks': False}, opts) + self.assertDictContainsSubset({'expose_tracebacks': 'false'}, opts) self.assertSequenceEqual(args, []) def test_cast_params(self): @@ -130,8 +146,8 @@ ]) self.assertDictContainsSubset({ 'host': 'localhost', - 'port': 80, - 'unix_socket_perms': 0o777, + 'port': '80', + 'unix_socket_perms':'777', }, opts) self.assertSequenceEqual(args, []) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/waitress-0.8.7/waitress/tests/test_buffers.py new/waitress-0.8.9/waitress/tests/test_buffers.py --- old/waitress-0.8.7/waitress/tests/test_buffers.py 2013-05-24 01:02:12.000000000 +0200 +++ new/waitress-0.8.9/waitress/tests/test_buffers.py 2014-05-16 23:14:59.000000000 +0200 @@ -105,10 +105,10 @@ inst.prune() self.assertTrue(inst.file is f) - def test__close(self): + def test_close(self): f = io.BytesIO() inst = self._makeOne(f) - inst._close() + inst.close() self.assertTrue(f.closed) class TestTempfileBasedBuffer(unittest.TestCase): @@ -149,13 +149,12 @@ from waitress.buffers import ReadOnlyFileBasedBuffer return ReadOnlyFileBasedBuffer(file, block_size) - def test_prepare_not_seekable_not_closeable(self): + def test_prepare_not_seekable(self): f = KindaFilelike(b'abc') inst = self._makeOne(f) result = inst.prepare() self.assertEqual(result, False) self.assertEqual(inst.remain, 0) - self.assertFalse(hasattr(inst, 'close')) def test_prepare_not_seekable_closeable(self): f = KindaFilelike(b'abc', close=1) @@ -163,7 +162,7 @@ result = inst.prepare() self.assertEqual(result, False) self.assertEqual(inst.remain, 0) - self.assertEqual(inst.close, f.close) + self.assertTrue(hasattr(inst, 'close')) def test_prepare_seekable_closeable(self): f = Filelike(b'abc', close=1, tellresults=[0, 10]) @@ -172,7 +171,7 @@ self.assertEqual(result, 10) self.assertEqual(inst.remain, 10) self.assertEqual(inst.file.seeked, 0) - self.assertFalse(hasattr(inst, 'close')) + self.assertTrue(hasattr(inst, 'close')) def test_get_numbytes_neg_one(self): f = io.BytesIO(b'abcdef') @@ -284,6 +283,17 @@ self.assertEqual(inst.buf.get(100), b'x' * 5) self.assertEqual(inst.strbuf, b'') + def test_append_with_len_more_than_max_int(self): + from waitress.compat import MAXINT + inst = self._makeOne() + inst.overflowed = True + buf = DummyBuffer(length=MAXINT) + inst.buf = buf + result = inst.append(b'x') + # we don't want this to throw an OverflowError on Python 2 (see + # https://github.com/Pylons/waitress/issues/47) + self.assertEqual(result, None) + def test_append_buf_None_not_longer_than_srtbuf_limit(self): inst = self._makeOne() inst.strbuf = b'x' * 5 @@ -373,6 +383,17 @@ inst.prune() self.assertNotEqual(inst.buf, buf) + def test_prune_with_buflen_more_than_max_int(self): + from waitress.compat import MAXINT + inst = self._makeOne() + inst.overflowed = True + buf = DummyBuffer(length=MAXINT+1) + inst.buf = buf + result = inst.prune() + # we don't want this to throw an OverflowError on Python 2 (see + # https://github.com/Pylons/waitress/issues/47) + self.assertEqual(result, None) + def test_getfile_buf_None(self): inst = self._makeOne() f = inst.getfile() @@ -386,19 +407,19 @@ f = inst.getfile() self.assertEqual(f, buf) - def test__close_nobuf(self): + def test_close_nobuf(self): inst = self._makeOne() inst.buf = None - self.assertEqual(inst._close(), None) # doesnt raise + self.assertEqual(inst.close(), None) # doesnt raise - def test__close_withbuf(self): + def test_close_withbuf(self): class Buffer(object): - def _close(self): + def close(self): self.closed = True buf = Buffer() inst = self._makeOne() inst.buf = buf - inst._close() + inst.close() self.assertTrue(buf.closed) class KindaFilelike(object): @@ -417,3 +438,16 @@ def tell(self): v = self.tellresults.pop(0) return v + +class DummyBuffer(object): + def __init__(self, length=0): + self.length = length + + def __len__(self): + return self.length + + def append(self, s): + self.length = self.length + len(s) + + def prune(self): + pass diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/waitress-0.8.7/waitress/tests/test_channel.py new/waitress-0.8.9/waitress/tests/test_channel.py --- old/waitress-0.8.7/waitress/tests/test_channel.py 2013-05-24 01:02:12.000000000 +0200 +++ new/waitress-0.8.9/waitress/tests/test_channel.py 2014-05-16 23:14:59.000000000 +0200 @@ -22,6 +22,18 @@ self.assertEqual(inst.addr, '127.0.0.1') self.assertEqual(map[100], inst) + def test_total_outbufs_len_an_outbuf_size_gt_sys_maxint(self): + from waitress.compat import MAXINT + inst, _, map = self._makeOneWithMap() + class DummyHugeBuffer(object): + def __len__(self): + return MAXINT + 1 + inst.outbufs = [DummyHugeBuffer()] + result = inst.total_outbufs_len() + # we are testing that this method does not raise an OverflowError + # (see https://github.com/Pylons/waitress/issues/47) + self.assertEqual(result, MAXINT+1) + def test_writable_something_in_outbuf(self): inst, sock, map = self._makeOneWithMap() inst.outbufs[0].append(b'abc') @@ -249,13 +261,33 @@ inst.logger = DummyLogger() def doraise(): raise NotImplementedError - inst.outbufs[0]._close = doraise + inst.outbufs[0].close = doraise result = inst._flush_some() self.assertEqual(result, True) self.assertEqual(buffer.skipped, 3) self.assertEqual(inst.outbufs, [buffer]) self.assertEqual(len(inst.logger.exceptions), 1) + def test__flush_some_outbuf_len_gt_sys_maxint(self): + from waitress.compat import MAXINT + inst, sock, map = self._makeOneWithMap() + class DummyHugeOutbuffer(object): + def __init__(self): + self.length = MAXINT + 1 + def __len__(self): + return self.length + def get(self, numbytes): + self.length = 0 + return b'123' + def skip(self, *args): pass + buf = DummyHugeOutbuffer() + inst.outbufs = [buf] + inst.send = lambda *arg: 0 + result = inst._flush_some() + # we are testing that _flush_some doesn't raise an OverflowError + # when one of its outbufs has a __len__ that returns gt sys.maxint + self.assertEqual(result, False) + def test_handle_close(self): inst, sock, map = self._makeOneWithMap() inst.handle_close() @@ -266,7 +298,7 @@ inst, sock, map = self._makeOneWithMap() def doraise(): raise NotImplementedError - inst.outbufs[0]._close = doraise + inst.outbufs[0].close = doraise inst.logger = DummyLogger() inst.handle_close() self.assertEqual(inst.connected, False) @@ -604,7 +636,7 @@ def __len__(self): return len(self.data) - def _close(self): + def close(self): self.closed = True class DummyAdjustments(object): @@ -660,7 +692,7 @@ def __init__(self): self.headers = {} - def _close(self): + def close(self): self.closed = True class DummyLogger(object): diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/waitress-0.8.7/waitress/tests/test_functional.py new/waitress-0.8.9/waitress/tests/test_functional.py --- old/waitress-0.8.7/waitress/tests/test_functional.py 2013-08-29 17:53:38.000000000 +0200 +++ new/waitress-0.8.9/waitress/tests/test_functional.py 2014-05-16 23:15:02.000000000 +0200 @@ -38,7 +38,10 @@ # Coverage doesn't see this as it's ran in a separate process. kw['port'] = 0 # Bind to any available port. super(FixtureTcpWSGIServer, self).__init__(application, **kw) - queue.put(self.socket.getsockname()) + host, port = self.socket.getsockname() + if os.name == 'nt': + host = '127.0.0.1' + queue.put((host, port)) class SubprocessTests(object): @@ -51,23 +54,25 @@ def start_subprocess(self, target, **kw): # Spawn a server process. - queue = multiprocessing.Queue() + self.queue = multiprocessing.Queue() self.proc = multiprocessing.Process( target=start_server, - args=(target, self.server, queue), + args=(target, self.server, self.queue), kwargs=kw, ) self.proc.start() if self.proc.exitcode is not None: # pragma: no cover raise RuntimeError("%s didn't start" % str(target)) # Get the socket the server is listening on. - self.bound_to = queue.get(timeout=5) + self.bound_to = self.queue.get(timeout=5) self.sock = self.create_socket() def stop_subprocess(self): if self.proc.exitcode is None: self.proc.terminate() self.sock.close() + # This give us one FD back ... + self.queue.close() def assertline(self, line, status, reason, version): v, s, r = (x.strip() for x in line.split(None, 2)) @@ -258,6 +263,8 @@ line, headers, response_body = read_http(fp) self.assertline(line, '200', 'OK', 'HTTP/1.1') self.assertEqual(response_body, b'') + self.assertEqual(headers['content-length'], '0') + self.assertFalse('transfer-encoding' in headers) def test_chunking_request_with_content(self): control_line = b"20;\r\n" # 20 hex = 32 dec @@ -277,6 +284,8 @@ line, headers, response_body = read_http(fp) self.assertline(line, '200', 'OK', 'HTTP/1.1') self.assertEqual(response_body, expected) + self.assertEqual(headers['content-length'], str(len(expected))) + self.assertFalse('transfer-encoding' in headers) def test_broken_chunked_encoding(self): control_line = "20;\r\n" # 20 hex = 32 dec @@ -1469,7 +1478,9 @@ try: response_line = fp.readline() except socket.error as exc: - if get_errno(exc) in (10053, 10054, 104): + fp.close() + # errno 104 is ENOTRECOVERABLE, In WinSock 10054 is ECONNRESET + if get_errno(exc) in (errno.ECONNABORTED, errno.ECONNRESET, 104, 10054): raise ConnectionClosed raise if not response_line: diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/waitress-0.8.7/waitress/tests/test_parser.py new/waitress-0.8.9/waitress/tests/test_parser.py --- old/waitress-0.8.7/waitress/tests/test_parser.py 2013-05-24 01:02:12.000000000 +0200 +++ new/waitress-0.8.9/waitress/tests/test_parser.py 2014-05-16 23:14:59.000000000 +0200 @@ -121,8 +121,8 @@ Transfer-Encoding: chunked X-Foo: 1 -20;\r -This string has 32 characters\r +20;\r\n +This string has 32 characters\r\n 0\r\n\r\n""" result = self.parser.received(data) self.assertEqual(result, 58) @@ -149,6 +149,23 @@ self.assertTrue(isinstance(self.parser.error, BadRequest)) + def test_received_chunked_completed_sets_content_length(self): + data = b"""\ +GET /foobar HTTP/1.1 +Transfer-Encoding: chunked +X-Foo: 1 + +20;\r\n +This string has 32 characters\r\n +0\r\n\r\n""" + result = self.parser.received(data) + self.assertEqual(result, 58) + data = data[result:] + result = self.parser.received(data) + self.assertTrue(self.parser.completed) + self.assertTrue(self.parser.error is None) + self.assertEqual(self.parser.headers['CONTENT_LENGTH'], '32') + def test_parse_header_gardenpath(self): data = b"""\ GET /foobar HTTP/8.4 @@ -168,7 +185,8 @@ self.assertEqual(self.parser.body_rcv, None) def test_parse_header_11_te_chunked(self): - data = b"GET /foobar HTTP/1.1\ntransfer-encoding: chunked" + # NB: test that capitalization of header value is unimportant + data = b"GET /foobar HTTP/1.1\ntransfer-encoding: ChUnKed" self.parser.parse_header(data) self.assertEqual(self.parser.body_rcv.__class__.__name__, 'ChunkedReceiver') @@ -183,15 +201,15 @@ self.parser.parse_header(data) self.assertEqual(self.parser.connection_close, True) - def test__close_with_body_rcv(self): + def test_close_with_body_rcv(self): body_rcv = DummyBodyStream() self.parser.body_rcv = body_rcv - self.parser._close() + self.parser.close() self.assertTrue(body_rcv.closed) - def test__close_with_no_body_rcv(self): + def test_close_with_no_body_rcv(self): self.parser.body_rcv = None - self.parser._close() # doesn't raise + self.parser.close() # doesn't raise class Test_split_uri(unittest.TestCase): @@ -389,5 +407,5 @@ def getbuf(self): return self - def _close(self): + def close(self): self.closed = True diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/waitress-0.8.7/waitress/tests/test_receiver.py new/waitress-0.8.9/waitress/tests/test_receiver.py --- old/waitress-0.8.7/waitress/tests/test_receiver.py 2013-05-24 01:02:12.000000000 +0200 +++ new/waitress-0.8.9/waitress/tests/test_receiver.py 2014-05-16 23:14:59.000000000 +0200 @@ -2,9 +2,9 @@ class TestFixedStreamReceiver(unittest.TestCase): - def _makeOne(self, buf, cl): + def _makeOne(self, cl, buf): from waitress.receiver import FixedStreamReceiver - return FixedStreamReceiver(buf, cl) + return FixedStreamReceiver(cl, buf) def test_received_remain_lt_1(self): buf = DummyBuffer() @@ -42,6 +42,11 @@ inst = self._makeOne(10, buf) self.assertEqual(inst.getbuf(), buf) + def test___len__(self): + buf = DummyBuffer(['1', '2']) + inst = self._makeOne(10, buf) + self.assertEqual(inst.__len__(), 2) + class TestChunkedReceiver(unittest.TestCase): def _makeOne(self, buf): @@ -142,13 +147,23 @@ inst = self._makeOne(buf) self.assertEqual(inst.getbuf(), buf) + def test___len__(self): + buf = DummyBuffer(['1', '2']) + inst = self._makeOne(buf) + self.assertEqual(inst.__len__(), 2) + class DummyBuffer(object): - def __init__(self): - self.data = [] + def __init__(self, data=None): + if data is None: + data = [] + self.data = data def append(self, s): self.data.append(s) def getfile(self): return self + + def __len__(self): + return len(self.data) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/waitress-0.8.7/waitress/tests/test_runner.py new/waitress-0.8.9/waitress/tests/test_runner.py --- old/waitress-0.8.7/waitress/tests/test_runner.py 2013-08-12 06:52:42.000000000 +0200 +++ new/waitress-0.8.9/waitress/tests/test_runner.py 2014-05-16 23:14:59.000000000 +0200 @@ -126,7 +126,7 @@ import waitress.tests.fixtureapps.runner as _apps def check_server(app, **kw): self.assertIs(app, _apps.app) - self.assertDictEqual(kw, {'port': 80}) + self.assertDictEqual(kw, {'port': '80'}) argv = [ 'waitress-serve', '--port=80', @@ -138,7 +138,7 @@ import waitress.tests.fixtureapps.runner as _apps def check_server(app, **kw): self.assertIs(app, _apps.app) - self.assertDictEqual(kw, {'port': 80}) + self.assertDictEqual(kw, {'port': '80'}) argv = [ 'waitress-serve', '--port=80', diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/waitress-0.8.7/waitress/tests/test_task.py new/waitress-0.8.9/waitress/tests/test_task.py --- old/waitress-0.8.7/waitress/tests/test_task.py 2013-08-29 17:53:38.000000000 +0200 +++ new/waitress-0.8.9/waitress/tests/test_task.py 2014-05-16 23:14:59.000000000 +0200 @@ -394,6 +394,16 @@ inst.channel.server.application = app self.assertRaises(AssertionError, inst.execute) + def test_preserve_header_value_order(self): + def app(environ, start_response): + write = start_response('200 OK', [('C', 'b'), ('A', 'b'), ('A', 'a')]) + write(b'abc') + return [] + inst = self._makeOne() + inst.channel.server.application = app + inst.execute() + self.assertTrue(b'A: b\r\nA: a\r\nC: b\r\n' in inst.channel.written) + def test_execute_bad_status_value(self): def app(environ, start_response): start_response(None, []) @@ -548,7 +558,7 @@ request.path = '' inst.request = request environ = inst.get_environment() - self.assertEqual(environ['PATH_INFO'], '/') + self.assertEqual(environ['PATH_INFO'], '') def test_get_environment_no_query(self): inst = self._makeOne() @@ -585,6 +595,16 @@ self.assertEqual(environ['PATH_INFO'], '/fuz') self.assertEqual(environ['SCRIPT_NAME'], '/foo') + def test_get_environ_with_url_prefix_empty_path(self): + inst = self._makeOne() + inst.channel.server.adj.url_prefix = '/foo' + request = DummyParser() + request.path = '/foo' + inst.request = request + environ = inst.get_environment() + self.assertEqual(environ['PATH_INFO'], '') + self.assertEqual(environ['SCRIPT_NAME'], '/foo') + def test_get_environment_values(self): import sys inst = self._makeOne() @@ -630,6 +650,85 @@ self.assertEqual(environ['wsgi.input'], 'stream') self.assertEqual(inst.environ, environ) + def test_get_environment_values_w_scheme_override_untrusted(self): + inst = self._makeOne() + request = DummyParser() + request.headers = { + 'CONTENT_TYPE': 'abc', + 'CONTENT_LENGTH': '10', + 'X_FOO': 'BAR', + 'X_FORWARDED_PROTO': 'https', + 'CONNECTION': 'close', + } + request.query = 'abc' + inst.request = request + environ = inst.get_environment() + self.assertEqual(environ['wsgi.url_scheme'], 'http') + + def test_get_environment_values_w_scheme_override_trusted(self): + import sys + inst = self._makeOne() + inst.channel.addr = ['192.168.1.1'] + inst.channel.server.adj.trusted_proxy = '192.168.1.1' + request = DummyParser() + request.headers = { + 'CONTENT_TYPE': 'abc', + 'CONTENT_LENGTH': '10', + 'X_FOO': 'BAR', + 'X_FORWARDED_PROTO': 'https', + 'CONNECTION': 'close', + } + request.query = 'abc' + inst.request = request + environ = inst.get_environment() + + # nail the keys of environ + self.assertEqual(sorted(environ.keys()), [ + 'CONTENT_LENGTH', 'CONTENT_TYPE', 'HTTP_CONNECTION', 'HTTP_X_FOO', + 'PATH_INFO', 'QUERY_STRING', 'REMOTE_ADDR', 'REQUEST_METHOD', + 'SCRIPT_NAME', 'SERVER_NAME', 'SERVER_PORT', 'SERVER_PROTOCOL', + 'SERVER_SOFTWARE', 'wsgi.errors', 'wsgi.file_wrapper', 'wsgi.input', + 'wsgi.multiprocess', 'wsgi.multithread', 'wsgi.run_once', + 'wsgi.url_scheme', 'wsgi.version']) + + self.assertEqual(environ['REQUEST_METHOD'], 'GET') + self.assertEqual(environ['SERVER_PORT'], '80') + self.assertEqual(environ['SERVER_NAME'], 'localhost') + self.assertEqual(environ['SERVER_SOFTWARE'], 'waitress') + self.assertEqual(environ['SERVER_PROTOCOL'], 'HTTP/1.0') + self.assertEqual(environ['SCRIPT_NAME'], '') + self.assertEqual(environ['HTTP_CONNECTION'], 'close') + self.assertEqual(environ['PATH_INFO'], '/') + self.assertEqual(environ['QUERY_STRING'], 'abc') + self.assertEqual(environ['REMOTE_ADDR'], '192.168.1.1') + self.assertEqual(environ['CONTENT_TYPE'], 'abc') + self.assertEqual(environ['CONTENT_LENGTH'], '10') + self.assertEqual(environ['HTTP_X_FOO'], 'BAR') + self.assertEqual(environ['wsgi.version'], (1, 0)) + self.assertEqual(environ['wsgi.url_scheme'], 'https') + self.assertEqual(environ['wsgi.errors'], sys.stderr) + self.assertEqual(environ['wsgi.multithread'], True) + self.assertEqual(environ['wsgi.multiprocess'], False) + self.assertEqual(environ['wsgi.run_once'], False) + self.assertEqual(environ['wsgi.input'], 'stream') + self.assertEqual(inst.environ, environ) + + def test_get_environment_values_w_bogus_scheme_override(self): + inst = self._makeOne() + inst.channel.addr = ['192.168.1.1'] + inst.channel.server.adj.trusted_proxy = '192.168.1.1' + request = DummyParser() + request.headers = { + 'CONTENT_TYPE': 'abc', + 'CONTENT_LENGTH': '10', + 'X_FOO': 'BAR', + 'X_FORWARDED_PROTO': 'http://p02n3e.com?url=http', + 'CONNECTION': 'close', + } + request.query = 'abc' + inst.request = request + self.assertRaises(ValueError, inst.get_environment) + class TestErrorTask(unittest.TestCase): def _makeOne(self, channel=None, request=None): @@ -737,6 +836,7 @@ host = '127.0.0.1' port = 80 url_prefix = '' + trusted_proxy = None class DummyServer(object): server_name = 'localhost' diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/waitress-0.8.7/waitress/utilities.py new/waitress-0.8.9/waitress/utilities.py --- old/waitress-0.8.7/waitress/utilities.py 2013-05-24 01:02:12.000000000 +0200 +++ new/waitress-0.8.9/waitress/utilities.py 2014-05-16 23:45:37.000000000 +0200 @@ -188,7 +188,11 @@ raise # pragma: no cover else: if stat.S_ISSOCK(st.st_mode): - os.remove(path) + try: + os.remove(path) + except OSError: # pragma: no cover + # avoid race condition error during tests + pass class Error(object): diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/waitress-0.8.7/waitress.egg-info/PKG-INFO new/waitress-0.8.9/waitress.egg-info/PKG-INFO --- old/waitress-0.8.7/waitress.egg-info/PKG-INFO 2013-08-29 17:56:58.000000000 +0200 +++ new/waitress-0.8.9/waitress.egg-info/PKG-INFO 2014-05-16 23:48:19.000000000 +0200 @@ -1,6 +1,6 @@ Metadata-Version: 1.1 Name: waitress -Version: 0.8.7 +Version: 0.8.9 Summary: Waitress WSGI server Home-page: https://github.com/Pylons/waitress Author: Chris McDonough @@ -16,6 +16,76 @@ http://docs.pylonsproject.org/projects/waitress/en/latest/ . + 0.8.9 (2014-05-16) + ------------------ + + - Fix tests under Windows. NB: to run tests under Windows, you cannot run + "setup.py test" or "setup.py nosetests". Instead you must run ``python.exe + -c "import nose; nose.main()"``. If you try to run the tests using the + normal method under Windows, each subprocess created by the test suite will + attempt to run the test suite again. See + https://github.com/nose-devs/nose/issues/407 for more information. + + - Give the WSGI app_iter generated when ``wsgi.file_wrapper`` is used + (ReadOnlyFileBasedBuffer) a ``close`` method. Do not call ``close`` on an + instance of such a class when it's used as a WSGI app_iter, however. This is + part of a fix which prevents a leakage of file descriptors; the other part of + the fix was in WebOb + (https://github.com/Pylons/webob/commit/951a41ce57bd853947f842028bccb500bd5237da). + + - Allow trusted proxies to override ``wsgi.url_scheme`` via a request header, + ``X_FORWARDED_PROTO``. Allows proxies which serve mixed HTTP / HTTPS + requests to control signal which are served as HTTPS. See + https://github.com/Pylons/waitress/pull/42. + + 0.8.8 (2013-11-30) + ------------------ + + - Fix some cases where the creation of extremely large output buffers (greater + than 2GB, suspected to be buffers added via ``wsgi.file_wrapper``) might + cause an OverflowError on Python 2. See + https://github.com/Pylons/waitress/issues/47. + + - When the ``url_prefix`` adjustment starts with more than one slash, all + slashes except one will be stripped from its beginning. This differs from + older behavior where more than one leading slash would be preserved in + ``url_prefix``. + + - If a client somehow manages to send an empty path, we no longer convert the + empty path to a single slash in ``PATH_INFO``. Instead, the path remains + empty. According to RFC 2616 section "5.1.2 Request-URI", the scenario of a + client sending an empty path is actually not possible because the request URI + portion cannot be empty. + + - If the ``url_prefix`` adjustment matches the request path exactly, we now + compute ``SCRIPT_NAME`` and ``PATH_INFO`` properly. Previously, if the + ``url_prefix`` was ``/foo`` and the path received from a client was ``/foo``, + we would set *both* ``SCRIPT_NAME`` and ``PATH_INFO`` to ``/foo``. This was + incorrect. Now in such a case we set ``PATH_INFO`` to the empty string and + we set ``SCRIPT_NAME`` to ``/foo``. Note that the change we made has no + effect on paths that do not match the ``url_prefix`` exactly (such as + ``/foo/bar``); these continue to operate as they did. See + https://github.com/Pylons/waitress/issues/46 + + - Preserve header ordering of headers with the same name as per RFC 2616. See + https://github.com/Pylons/waitress/pull/44 + + - When waitress receives a ``Transfer-Encoding: chunked`` request, we no longer + send the ``TRANSFER_ENCODING`` nor the ``HTTP_TRANSFER_ENCODING`` value to + the application in the environment. Instead, we pop this header. Since we + cope with chunked requests by buffering the data in the server, we also know + when a chunked request has ended, and therefore we know the content length. + We set the content-length header in the environment, such that applications + effectively never know the original request was a T-E: chunked request; it + will appear to them as if the request is a non-chunked request with an + accurate content-length. + + - Cope with the fact that the ``Transfer-Encoding`` value is case-insensitive. + + - When the ``--unix-socket-perms`` option was used as an argument to + ``waitress-serve``, a ``TypeError`` would be raised. See + https://github.com/Pylons/waitress/issues/50. + 0.8.7 (2013-08-29) ------------------ -- To unsubscribe, e-mail: opensuse-commit+unsubscr...@opensuse.org For additional commands, e-mail: opensuse-commit+h...@opensuse.org
