[Web-SIG] [Python-Dev] Adding wsgiref to stdlib
Hello all, I've been following the discussion around adding wsgiref to the stdlib and it sounds like a very good idea. However I'm a little concerned as it seems only wsgiref has been suggested to be included. I wonder if you guys intend to review other implementations before going ahead? I ask this because the simple_server.py module of wsgiref says: """It has not been reviewed for security issues, however, and we strongly recommend that you use a "real" web server for production use. """ Therefore, I wonder what is the final purpose of such addition? Is it merely to have default WSGI implementation that *might* not be scalable in production? I have nothing against wsgiref mind you. I'm fairly sure Phillip has done a great job but I simply wanted to know if you would consider checking other implemetations. - Sylvain ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] [Python-Dev] Adding wsgiref to stdlib
> Anything that could be considered of sufficiently industrial strength > to be secure and scalable in production would necessarily be such a > large project, such a complex code base, and have such different > release cycle that it would not make a good standard library > candidate. (Think mod_python, Twisted, Zope, Apache; think tail > wagging the doc.) I was thinking into products that have shown good results over the last past year and are far less complex than the ones you mention. There are existing implementations that could be easily extracted from their environment and might be better: CherryPy, Paste, Pylons, Colubrid, etc. all offer what wsgiref provides (well to what I understand from wsgiref code, Phillip could correct my knowledge here). Nevertheless, it might be more useful to define the boundaries of including a WSGI implementation to the stdlib before even choosing one. AFAIK, WSGI is splitted into three distinct layers. A web server Some middlewares An application We will not include middlewares I believe. I'm not sure I would see the point of adding a default WSGI application. So we're left with the web server part to test and then see which is the one filling the best the criteria folks around here may decide to discriminate on. - Sylvain ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
[Web-SIG] wsgiref bug with HEAD request
All, It seems the default server from wsgiref (from wsgiref.simple_server import make_server) seems not to respect Content-Length in case of HEAD request. Since no body can be returned in a response to a HEAD request, the content length is set to 0 by the server. In that case Content-Length is therefore set by the application or a middleware. wsgiref server disregard the existing value and sets to 0 either way. Seems bogus to me or am I missing something here? - Sylvain ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] routing_args accepted - selector updated
Luke Arno wrote:
> I see that Ian has marked the proposed routing args
> convention as accepted. I have updated selector and
> now consider it a stable feature so feel free to use it.
>
> http://wsgi.org/wsgi/Specifications/routing_args
> https://lukearno.com/projects/selector/
>
Hello there,
Sorry for being so late into the discussion. I must say I like that
proposal and I completely sedond it. Except for one small point, the
spec says:
"The values in routing_args need not be strings (except for the keys of
named_args). For instance, a dispatcher is allowed to parse
/archive/2005/10/01 into ((), {'date': datetime.date(2005, 10, 1)})."
This could be an issue for deployment if we consider that applications
or other middleware will not be able to know what a middleware that
implements this proposal has decided to do, map the values or not.
Say I have the value '1' in my URI, I don't want the dispatcher to
assume it's an int and maps it to '1'. Maybe this value is an identifier
for me and I need the complete value. There is a loss of information.
What I would like to add to the proposal if it's not too late is the
mean to tell the middleware whether or not I agree that it maps values
automatically.
Say:
s = Selector(allow_mapping=False)
Or something equivlent.
Personally I would in fact disable that feature altogether and let the
application decide what is best but I doubt you will appreciate that idea.
In any case I will certainly use this proposal in the WSGI dispatcher of
Amplee, my APP implementation.
http://trac.defuze.org/browser/oss/amplee/amplee/handler/store/wsgi/__init__.py
Thanks,
- Sylvain
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe:
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] New spec: simple authentication
Ian,
Why disallowing the application to set the WWW-Authenticate header? If a
middleware is present it will be overwritten anyway. If no middleware is
there then at least you won't break the first MUST in section 10.4.2 of
RFC 2616.
What happens if part of my application is to be protected by OpenID,
another by Digest and a third one by a in-house auth scheme? How does your
spec. deal with this?
- Sylvain
> I added a new spec describing authentication middleware with WSGI. It
> doesn't describe anything new, really, it just describes what I think is
> the most basic best practice in doing WSGI-based authentication
> middleware:
>
>http://wsgi.org/wsgi/Specifications/simple_authentication
>
> I offer it more as a basis for other specifications to build upon.
>
> Also copied below:
>
> :Title: Simple Authentication
> :Author: Ian Bicking <[EMAIL PROTECTED]>
> :Discussions-To: Python Web-SIG
> :Status: Proposed
> :Created: 13-Nov-2006
>
> .. contents::
>
> Abstract
>
>
> This describes a simple pattern for implementing authentication in WSGI
> middleware. This does not propose any new features or environment keys;
> it only describes a baseline recommended practice.
>
> Rationale
> -
>
> Authentication is probably the most common detail that should be
> abstracted away from an application, as it is a concern most often bound
> to a *deployment*.
>
> Specification
> -
>
> There are two components to authentication:
>
> 1. Indicating when a request is authenticated, and by who
> 2. Responding that authentication is necessary
>
> There are already two conventions for this:
>
> 1. Put the username in ``REMOTE_USER``
> 2. Respond with ``401 Unauthorized``
>
> .. note::
> Please do not confused ``401 Unauthorized`` with "permission
> denied". Permission denied should be indicated with ``403 Forbidden``.
>
> ``REMOTE_USER``:
> This should be the string username of the user, nothing more.
> ``401 Unauthorized``:
> Because middleware is handling the authentication, additional
> information is not required. You do not (and should not) include a
> ``WWW-Authenticate`` header. The middleware may include that header, or
> may change the response in some other way to handle the login.
>
> Example
>
>
> The first example implements simple HTTP Basic authentication::
>
>class HTTPBasic(object):
>
>def __init__(self, app, user_database, realm='Website'):
>self.app = app
>self.user_database = user_database
>
>def __call__(self, environ, start_response):
>def repl_start_response(status, headers, exc_info=None):
>if status.startswith('401'):
>remove_header(headers, 'WWW-Authenticate')
>headers.append(('WWW-Authenticate', 'Basic
> realm="%s"' % self.realm))
>return start_response(status, headers)
>auth = environ.get('HTTP_AUTHORIZATION')
>if auth:
>scheme, data = auth.split(None, 1)
>assert scheme.lower() == 'basic'
>username, password = data.decode('base64').split(':', 1)
>if self.user_database.get(username) != password:
>return self.bad_auth(environ, start_response)
>environ['REMOTE_USER'] = username
>del environ['HTTP_AUTHORIZATION']
>return self.app(environ, repl_start_response)
>
>def bad_auth(self, environ, start_response):
>body = 'Please authenticate'
>headers = [
>('content-type', 'text/plain'),
>('content-length', str(len(body))),
>('WWW-Authenticate', 'Basic realm="%s"' % self.realm)]
>start_response('401 Unauthorized', headers)
>return [body]
>
>def remove_header(headers, name):
>for header in headers:
>if header[0].lower() == name.lower():
>headers.remove(header)
>break
>
> Problems
>
>
> * Strictly speaking, it is illegal to send a ``401 Unauthorized``
> response without the WWW-Authenticate header. If no middleware is
> installed, most browsers will treat it like a ``200 OK``. There is also
> no way to detect if an appropriate middleware is installed.
>
> * This doesn't give any other information about the user. That
> information can go in other keys, but that is not addressed in this
> specification currently.
>
> * Some login methods will redirect the user, and any POST request data
> will possibly be lost. (Note that a specification like
> ["handling_post_forms"] helps address this problem.)
>
> Other Possibilities
> ---
>
> * While you can add to this specification, I think it's the most logical
> and useful way to do authentication and better efforts can build on this
> base.
>
> Open Issues
> ---
>
> See Problems.
> ___
> W
[Web-SIG] ANN: amplee 0.3.6 - Atom Publishing Protocol implementation
Hi all, I'm glad to announce the release of amplee in version 0.3.6 This release is a bug fixes but with a few new features as well: * Fixed the AtomMember class * Refactored most of the member internal code * Fixed a few issues with both the CherryPy and pure WSGI handlers * Added new examples but separated them from the main distribution. * Fixed the XHTML member class * Fixed the storage API issue when listing resources * Fixed a bug when checking for the existing of a resource * Moved to support bridge 0.2.1 * Started to add unit tests * Removed the genser dependency The API should not have changed a lot apart from specific cases. I hope this won't break too much code. I would like to thank David Turner for his great feedback. == Download == * easy_install -U amplee * Tarballs http://www.defuze.org/oss/amplee/ * svn co https://svn.defuze.org/oss/amplee/ == Documentation == http://trac.defuze.org/wiki/amplee http://www.defuze.org/oss/amplee/doc/html/ == Examples == You can get some source code examples at http://defuze.org/oss/amplee/amplee-examples-0.3.6.tgz == TODO == * Add more tests * Improve documentation Have fun, -- Sylvain Hellegouarch http://www.defuze.org Reply ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] wsgiref questions
>> Well, we're using wsgiref's simple_server which only speaks HTTP/1.0, >> but we really need to speak HTTP/1.1 and use chunked. Where do you >> recommend we put the chunking instead? If you are not to fuss about which WSGI server you can afford to use, you could grab the server from CherryPy 3.0 which is HTTP/1.1 fluent. from cherrypy import wsgiref [1] You'd be done. - Sylvain [1] http://www.cherrypy.org/browser/trunk/cherrypy/wsgiserver.py ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] wsgiref questions
Sylvain Hellegouarch wrote:
>>> Well, we're using wsgiref's simple_server which only speaks HTTP/1.0,
>>> but we really need to speak HTTP/1.1 and use chunked. Where do you
>>> recommend we put the chunking instead?
>
>
> If you are not to fuss about which WSGI server you can afford to use,
> you could grab the server from CherryPy 3.0 which is HTTP/1.1 fluent.
>
> from cherrypy import wsgiref [1]
>
> You'd be done.
>
Because an example is worth all the words:
from cherrypy import wsgiserver
def my_crazy_app(environ, start_response):
status = '200 OK'
response_headers = [('Content-type','text/plain')]
start_response(status, response_headers)
return ['Hello world!\n']
# This won't call CherryPy machinery at all
# Only the WSGI server which is independant from CherryPy itself
# Its name simply reflects its origin...
server = wsgiserver.CherryPyWSGIServer(('localhost', 8070), [('/',
my_crazy_app)], numthreads=10, server_name='localhost')
# Want SSl support? Just set those attributes
# server.ssl_certificate = ...
# server.ssl_private_key = ...
if __name__ == '__main__':
server.start()
The nice thing is that you can also support many WSGI applications in
one instance of your server. Anyway only for those who'd be interested ;)
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe:
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] wsgiref questions
> We decided to add chunking encoding to our own server, it wasn't all > that hard. Hopefully you will release this code as part of wsgiref and let the community benefit from it, right? ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] wsgiref questions
> On 12/22/06, Sylvain Hellegouarch <[EMAIL PROTECTED]> wrote: >> > We decided to add chunking encoding to our own server, it wasn't all >> > that hard. >> >> Hopefully you will release this code as part of wsgiref and let the >> community benefit from it, right? > > We didn't modift wsgiref, we added it to a proprietary module; Makes sense that you can't release it then :) and I > don't think our code has anything to offer over what's already > available through other channels, such as the CherryPy server. True but the thing is that it's wsgiref which belongs to the stdlib not CherryPy, it'd be nice if the reference WSGI server could be a little more complete I suppose. Besides for some obscure reason some people have a bad opinion of CP. > > (Also, wsgiref violates a couple of Python style guides that make me > not want to update it myself. Phillip promised he would clean it up > for distribution but never did, so the version distributed with Python > 2.5 has a few strange ideosyncracies that I'm afraid to clean up > because last time someone touched Phillip's code he threw a fit.) Oh good then that I didn't try to understand the Content-Length bug I raised a few months ago on HEAD requests :) Thanks for the feedback by the way. Much appreciated. - Sylvain ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] wsgiref questions
> On 12/22/06, Sylvain Hellegouarch <[EMAIL PROTECTED]> wrote: >> > We decided to add chunking encoding to our own server, it wasn't all >> > that hard. >> >> Hopefully you will release this code as part of wsgiref and let the >> community benefit from it, right? > > We didn't modift wsgiref, we added it to a proprietary module; and I > don't think our code has anything to offer over what's already > available through other channels, such as the CherryPy server. > > (Also, wsgiref violates a couple of Python style guides that make me > not want to update it myself. Phillip promised he would clean it up > for distribution but never did, so the version distributed with Python > 2.5 has a few strange ideosyncracies that I'm afraid to clean up > because last time someone touched Phillip's code he threw a fit.) > One question popped into my mind. One of the core idea behind WSGI is the pluggability of components respecting the interface proposed by PEP 333. I wonder if there was reason not to try a different WSGI server when facing a limitation with wsgiref rather than using a proprietary module? CherryPy server or another does not matter. Of course since I have no clue of your infrastructure and design I'm not judging here but your feedback would be interesting on that particular case. I'm intrigued :) Thanks, - Sylvain ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
[Web-SIG] ANN: amplee 0.4.0
Hi all, I finally released a new version of amplee. I've moved from 0.3.x to 0.4.x as there are a couple of modification to the API that were worse the bump. I think this version is much more stable and bug free. Mind you it's a long way before I can claim is entirely unit tested. But it gets there. The main modifications since 0.3.6 are: * Added a loader feature. I realized that setting up the store was a recurrent task and I wondered how to help in this task. I came up with the loader feature. Basically you describe your APP store within a config file (pure INI) and calls the loader method. This will construct entirely your store and returns it to you. This is quite handy and makes the creation of a store much easier. * Handler API introduced. Amplee does the best it can to provide you with an API an tools to handle the dirty work of APP and let you enhance it by a callback system. In previous version those callbacks were attached to the collection which forced to some not very friendly hacks. Now it's a matter of creating a class that implements a set of methods which will then be called by amplee at the right moment. This class is what I call a handler and associated with a media-type that the collection accepts. * The loading and reloading of members is more flexible. In the past you could solely reload all members of a store or none. Now you have more granularity upon what should be loaded into a collection's cache. * Many notable bugs have been fixed in the handling of Atom within the members and they should be much more reliable now. * You can now find a small blog example that shows you how to use amplee. If you think of upgrading you should note that because of the modification on the callback API you may have some work to do. But this should not be too difficult. == Download == * easy_install -U amplee * Tarballs http://www.defuze.org/oss/amplee/ * svn co https://svn.defuze.org/oss/amplee/ == Documentation == http://trac.defuze.org/wiki/amplee http://www.defuze.org/oss/amplee/doc/html/ == Examples == You can get some source code examples at http://defuze.org/oss/amplee/amplee-example-0.4.0.tgz == TODO == * Add more tests * Improve documentation * Test with IronPython * Enhance WSGI support Have fun, -- Sylvain Hellegouarch http://www.defuze.org Reply ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] The web dudes pad is open for business
Titus Brown wrote: > now doesn't everyone wish they were at PyCon, too? ;) > Precisely what I told myself :) ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Newline values in WSGI response header values.
> Can anyone confirm for me what the behaviour should be if someone > includes a newline in the value of a WSGI response header? > > CGI specification would seem to disallow it and thus WSGI adapter > should by rights possibly produce an error if user code does it. > > At the moment I know of no WSGI adapter implementation which validates > whether a newline appears in the value of a WSGI response header. For > many WSGI adapters this means that a header of: > > Key1: "Value1\r\nKey2: Value2" > > will actually translate into two separate headers being sent back to > client. > > For a header of: > > Key3: "Value3a\r\nValue3b" > > in a WSGI adapter which simply passes things through, the client would > get an invalid header line, which in general it would ignore. If > however this was generated when hosted with a CGI-WSGI adapter, for > Apache at least, Apache would generate a 500 error itself due to > detected a header line of invalid format. > > Thus, is an embedded newline in value invalid? Would it be reasonable > for a WSGI adapter to flag it as an error? > I might be reading the spec wrong but it doesn't seem to be forbidden by RFC 2616. Section 4.2 says: > Any LWS that occurs between field-content MAY be replaced with a single SP before interpreting the field value or forwarding the message downstream. Then a look at the definition of separators shows us that SP is a valid separator. Since section 2.1 tells: > Except where noted otherwise, linear white space (LWS) can be included between any two adjacent words (token or quoted-string), and between adjacent words and separators, without changing the interpretation of a field. It sounds to me that this is a valid construct but a WSGI adapter might consider converting those CRLF into simple SP as said in 2.1 again: > A recipient MAY replace any linear white space with a single SP before interpreting the field value or forwarding the message downstream. - Sylvain -- Sylvain Hellegouarch http://www.defuze.org ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Newline values in WSGI response header values.
> 2008/6/12 Sylvain Hellegouarch <[EMAIL PROTECTED]>: >> >>> Can anyone confirm for me what the behaviour should be if someone >>> includes a newline in the value of a WSGI response header? >>> >>> CGI specification would seem to disallow it and thus WSGI adapter >>> should by rights possibly produce an error if user code does it. >>> >>> At the moment I know of no WSGI adapter implementation which validates >>> whether a newline appears in the value of a WSGI response header. For >>> many WSGI adapters this means that a header of: >>> >>> Key1: "Value1\r\nKey2: Value2" >>> >>> will actually translate into two separate headers being sent back to >>> client. >>> >>> For a header of: >>> >>> Key3: "Value3a\r\nValue3b" >>> >>> in a WSGI adapter which simply passes things through, the client would >>> get an invalid header line, which in general it would ignore. If >>> however this was generated when hosted with a CGI-WSGI adapter, for >>> Apache at least, Apache would generate a 500 error itself due to >>> detected a header line of invalid format. >>> >>> Thus, is an embedded newline in value invalid? Would it be reasonable >>> for a WSGI adapter to flag it as an error? >>> >> >> I might be reading the spec wrong but it doesn't seem to be forbidden by >> RFC 2616. >> >> Section 4.2 says: >> >>> Any LWS that occurs between field-content MAY be replaced with a single >> SP before interpreting the field value or forwarding the message >> downstream. >> >> Then a look at the definition of separators shows us that SP is a valid >> separator. >> >> Since section 2.1 tells: >> >>> Except where noted otherwise, linear white space (LWS) can be included >> between any two adjacent words (token or quoted-string), and between >> adjacent words and separators, without changing the interpretation of a >> field. >> >> It sounds to me that this is a valid construct but a WSGI adapter might >> consider converting those CRLF into simple SP as said in 2.1 again: >> >>> A recipient MAY replace any linear white space with a single SP before >> interpreting the field value or forwarding the message downstream. > > A LWS is: > > LWS= [CRLF] 1*( SP | HT ) > > Ie, not just a single CRLF, but a CRLF followed by a space or tab. > > Thus, can't just replace CRLF only with a space. > > Anyway, the wording of my question and reference to CGI was a bit > wrong, as WSGI response headers are probably more governed by HTTP > RFC. > > To clarify, what we really have is two cases, the first is return of a > value with a valid LWS as specified by HTTP RFC. > > If the WSGI adapter is mapping direct to HTTP, then it can pass it > straight through. If however the WSGI adapter hosts on top a interface > with CGI like semantics, then it should translate LWS to single space > as described. > > The second case is an embedded CRLF which isn't followed by space or > tab and thus isn't a LWS. This is the case which causes problems and > am asking whether it should be detected and flagged as an errornous > response. > You might want to take the question to the HTTP-BIS charter and follow-up on that issue: http://tools.ietf.org/wg/httpbis/trac/ticket/30 - Sylvain -- Sylvain Hellegouarch http://www.defuze.org ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Python 3.0 and WSGI 1.0.
Hi All, > Is there going to be any simple answer to all of this? :-( > Would there be any interest in asking the HTTP-BIS working group [1] what they think about it? Currently I couldn't find anything in their drafts suggesting they had decided to clarify this issue from a protocol's perspective but they might consider it to be relevant to their goals. - Sylvain [1] http://www.ietf.org/html.charters/httpbis-charter.html -- Sylvain Hellegouarch http://www.defuze.org ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Python 3.0 and WSGI 1.0.
Probably of interest in regards to this discussion: http://lists.w3.org/Archives/Public/ietf-http-wg/2009AprJun/0057.html http://trac.tools.ietf.org/wg/httpbis/trac/ticket/63 This applies to headers but probably shows that RFC 2047 is gradually ruled out of HTTP. - Sylvain -- Sylvain Hellegouarch http://www.defuze.org ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Future of WSGI
I'm not aware of any problems with representing the request with a dictionary. Can you give examples? Though it shouldn't be considered as a problem, the fact that probably no existing framework actually use the raw dictionary (there is, in almost all cases, a wrapping into a friendlier object), one might wonder why keeping such a low level interface rather than directly provide a higher level interface is a good idea. After all creating those dictionaries for no good reason aside from sending them to the next layer which will map them into a WebOb, a yaro, a cherrypy request, or zope request, etc. seems slightly pointless (I'm not versed into Python internals, but doesn't it have also a cost of creating rather useless objects repeatedly like that?) I know WSGI tries hard not to force into one implementation but still... - Sylvain ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Future of WSGI
Henry Precheur a écrit : On Tue, Nov 24, 2009 at 11:16:05PM +0100, Sylvain Hellegouarch wrote: Though it shouldn't be considered as a problem, the fact that probably no existing framework actually use the raw dictionary (there is, in almost all cases, a wrapping into a friendlier object), one might wonder why keeping such a low level interface rather than directly provide a higher level interface is a good idea. After all creating those dictionaries for no good reason aside from sending them to the next layer which will map them into a WebOb, a yaro, a cherrypy request, or zope request, etc. seems slightly pointless 1. Would you say that POSIX is useless because there are lots of libraries and applications build on top of it? Why not implement those libraries and applications directly without using POSIX? If I'm not mistaken that's what people do when they want performances rather than portability. But point taken. 2. Guess what: WebOb, Werkzeug, Yaro, Django, CherryPy, and the others have a different interfaces for their Request/Response objects. Because for Request/Response there's hardly one-size fits all. There's certainly some common ground, but every framework has different needs. Well thank you for the reminder but I kind of knew that ;) It doesn't mean it's neither elegant nor efficient to create such a low level object. (I'm not versed into Python internals, but doesn't it have also a cost of creating rather useless objects repeatedly like that?) The dictionary is passed as a reference like every Python objects. So it doesn't cost anything to use it instead of an object. I talked about object creation not object passing. - Sylvain ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Future of WSGI
Please replace '...' with actual code or at least some description of what it's doing. Lots of people have been trying to define a nice interface for these objects for YEARS. People who know a great deal about HTTP, and Python. And yet there's not a single implementation that's widely accepted as the "best of breed". Mostly because no such discussion ever took place. Everyone does its own recipe and yet most request interface actually offer a very similar API. If it wasn't for WSGI, most frameworks wouldn't even talk to each other yet. But since it's the time of "what could be improved", it seemed right to suggest to do better there too. Now I don't have a proposal so I wouldn't be upset if the community simply says no (I can understand Ian's point in an earlier response) but the question is rather valid nonetheless. - Sylvain ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Future of WSGI
> 2009/11/25 Henry Precheur : >> Please replace '...' with actual code or at least some description of >> what it's doing. Lots of people have been trying to define a nice >> interface for these objects for YEARS. People who know a great deal >> about HTTP, and Python. And yet there's not a single implementation >> that's widely accepted as the "best of breed". > > class Request(object): > def __init__(self, stream): > self.environ = read_headers_until_crlf(stream) > self.stream = stream > > These headers are then "general-header", "request-header", > "entity-header". The stream is what remains. > Personally, I would favor the idea that WSGI2 specifies the way headers should be mapped to object attributes (e.g. Content-Type would become content_type) and then let duck typing magic happen rather than specifying a class from which to inherit for instance. Instead of a dictionary, you'd provide an object that maps headers and a few other attributes accordingly. But again, it's just wishful thinking ;) - Sylvain -- Sylvain Hellegouarch http://www.defuze.org ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Future of WSGI
Robert Brewer a écrit : Sylvain Hellegouarch wrote: Personally, I would favor the idea that WSGI2 specifies the way headers should be mapped to object attributes (e.g. Content-Type would become content_type) and then let duck typing magic happen rather than specifying a class from which to inherit for instance. How would you handle HTTP extension headers like X-MyEnterprise-Metadata? x_myenteprise_metadata Now I get this only makes sense where the header is valid as a Python identifier so more limited than a dict key for sure. Cook [1] might be appropriate to read here: "...abstract data types facilitate adding new operations, while [objects] facilitate adding new representations... Abstract data types define operations that collect together the behaviors for a given action. Objects organize the matrix the other way, collecting together all the actions associated with a given representation. It is easier to add new operations in an ADT, and new representations using objects." But that's my point, we discuss request representation, not behavior. IMO, it's quite appropriate that we essentially use an ADT (a dict) at the lowest level, precisely because it constrains the representation. This is the essence of The Zen of CherryPy #8 "Subclassed builtins are better than custom types" (really, custom _classes_) and #9 "But builtin types are even better". People can then objectify those ADTs to their representational taste. That's fine but looks redundant eventually in my opinion. - Sylvain ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] PEP 444 / WSGI 2 Async
2011/1/6 Alex Grönholm > 06.01.2011 20:02, Eric Larson kirjoitti: > >> At Thu, 6 Jan 2011 13:03:15 + (GMT), >> chris dent wrote: >> >>> >>> On async: >>> >>> I agree with some others who have suggested that maybe async should be >>> its own thing, rather than integrated into a WSGI2. A server could >>> choose to be WSGI2 compliant or AWSGI compliant, or both. >>> >>> >> +1 >> >> After seeing some of the ideas regarding how to add async into a new >> version of WSGI, it isn't the specific problem the async feature >> addresses in terms of WSGI servers. Is the goal is to support long >> running connections? Are we trying to support WebSockets and other >> long running connection interfaces? If that is the case, async is a >> *technique* for handling this paradigm, but it doesn't address the >> real problem. There are techniques that have sounded reasonable like >> making available the socket such that a server can give it to the >> application to do something use with it (here is an example doing >> something similar with CherryPy - >> http://trac.defuze.org/browser/oss/ws4cp/ws4cp.py). >> > The primary idea behind asynchronous servers/applications is the ability to > efficiently serve a huge number of concurrent connections with a small > number of threads. Asynchronous applications tend to be faster because there > is less thread context switching happening in the CPU. Any application that > runs on top of a web server that allocates less threads to the application > than the number of connections has to be quick to respond so as not to > starve the thread pool or block the event loop. This is true regardless of > whether nonblocking I/O or some other technique is used. I'm a bit unclear > as to how else you would do this. Care to elaborate on that? I looked at the > Cherrypy code, but I couldn't yet figure that out. Since I wrote that piece of code, I guess I ought to chime in. First of all, the code isn't part of CherryPy, simply it's one idea to provide WebSocket to CherryPy. Considering WebSocket bootstraps on HTTP but once that's done, it's just a raw socket with bits and pieces on top, I wanted to find a way not to block CherryPy from serving other requests once a WebSocket handshake had been performed. The idea was simply to detach the socket from the worker thread once the handshake had been performed. Then the application had a socket at hand and this particular instance, I simply decided to use asyncore to loop through those sockets aside from the CherryPy HTTP server. In effect, you end up with asyncore for WS sockets and CherryPy for any HTTP serving but from within one single process, using CherryPy's main loop. By large this is not a generic solution for implementing async in WSGI but a specific example on how one can have both threads and an async loop playing together. It's merely a proof of concept :) Hope that clarifies that piece of code. -- - Sylvain http://www.defuze.org http://twitter.com/lawouach ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] A 'shutdown' function in WSGI
> > > Yes, I also think shutting down the server is completely orthogonal to > requests. > > CherryPy is using an approach where, indeed, both are orthogonal. Tools [1], which are similar to middlewares, are dealing with requests whereas the CherryPy engine [2] is the basis upon which the whole application and HTTP servers are running. The engine provides, amongst others, a clean way to stop and/or exit the whole stack independently from running requests. However, due to the way Robert Brewer designed it, it also means this can, if wanted, be performed from a request handler at any time. Note that the engine architecture, a bus, allows obviously for functions to subscribe at the time the engine shuts down in order to perform further operations that you would require. Robert has also started the work of extracting the engine from CherryPy itself for other frameworks to rely on [3]. -- - Sylvain http://www.defuze.org http://twitter.com/lawouach [1] http://docs.cherrypy.org/stable/concepts/tools.html [2] http://docs.cherrypy.org/stable/concepts/engine.html [3] https://bitbucket.org/cherrypy/magicbus ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] A 'shutdown' function in WSGI
On Tue, Feb 21, 2012 at 10:31 AM, Graham Dumpleton < graham.dumple...@gmail.com> wrote: > On 21 February 2012 20:26, Simon Sapin wrote: > > Le 21/02/2012 09:23, Tarek Ziadé a écrit : > > > >>Instead of having to provide two or three objects separately to a > >>server, how about making the callbacks attributes of the application > >>callable? > >> > >> > >> can you show us an example ? > > > > > > Proposal: > > > > Function-based: > > > >def startup(): > >return open_resource(something) > > > >def shutdown(resource): > >resource.close() > > > >def application(environ, start_response): > ># ... > >return response_body > > > >application.startup = startup > >application.shutdown = shutdown > > > > Class-based: > > > >class App(object): > >def startup(self): > >return open_resource(something) > > > >def shutdown(self, resource): > >resource.close() > > > >def __call__(self, environ, start_response): > ># ... > >return response_body > > > >application = App() > > > > The return value of startup() can be any python object and is opaque to > the > > server. It is passed as-is to shutdown() > > > > startup() could take more parameters. Maybe the application (though can > we > > already have it as self for class-based or in a closure for > function-based) > > You do realise you are just reinventing context managers? > > With this 'application' do requests. > > But then it was sort of suggested that was a bit too radical idea when > I have mentioned viewing it that way before. :-( > > One might wonder if having access to process management should be part of WSGI in the first place. -- - Sylvain http://www.defuze.org http://twitter.com/lawouach ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] websocket support in WSGI
> > > In other words: The responsibility for the connection (and socket) is > passed to the application. > > This works well with traditional threaded servers. The application can > spawn a new worker thread, put the job into a queue or whatever and then > return from the application callable, allowing the server thread to > continue handling new connections. > > This is exactly how ws4py was implemented when using CherryPy for the HTTP server performing the handshake [1]. There's also a WSGI middleware [2] but it's heavily geared towards gevent and may not be reusable easily elsewhere I'm afraid. -- - Sylvain http://www.defuze.org http://twitter.com/lawouach [1] https://github.com/Lawouach/WebSocket-for-Python/blob/master/ws4py/server/cherrypyserver.py [2] https://github.com/Lawouach/WebSocket-for-Python/blob/master/ws4py/server/wsgi/middleware.py ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] websocket support in WSGI
On Tue, May 22, 2012 at 3:39 PM, Alex Grönholm wrote: > 22.05.2012 10:38, Sylvain Hellegouarch kirjoitti: > > >> In other words: The responsibility for the connection (and socket) is >> passed to the application. >> >> This works well with traditional threaded servers. The application can >> spawn a new worker thread, put the job into a queue or whatever and then >> return from the application callable, allowing the server thread to >> continue handling new connections. >> >> > This is exactly how ws4py was implemented when using CherryPy for the > HTTP server performing the handshake [1]. There's also a WSGI middleware > [2] but it's heavily geared towards gevent and may not be reusable easily > elsewhere I'm afraid. > > It's also a hack that violates the WSGI spec. It's also not usable through > reverse proxying or FCGI/SCGI. > Yeap that's very true and it'll stay a hack until WSGI is updated to support it or explicitely reject protocols such as WebSocket. In the meanwhile, I'm personally fine having projects like ws4py to play with and decide what could work and what couldn't. -- - Sylvain http://www.defuze.org http://twitter.com/lawouach ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] WSGI for HTTP/2.0 ?
Hi Benoit, > I actually wonder if websockets or other asynchronous protocols should be > handled by the new WSGI SPEC. Shouldn't we just standardize the way the > socket is given to another library? > Considering the websocket connection is initiated via a HTTP request, it would be a good idea that the newer WSGI interface keeps it in mind even if it doesn't address the protocol itself. As you say, the current state is akward at best. -- - Sylvain http://www.defuze.org http://twitter.com/lawouach ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Pre-PEP: The WSGI Middleware Escape for Native Server APIs
Hi, 2014-09-30 1:19 GMT+02:00 PJ Eby : > Per the previous discussion about HTTP/2, websockets, et al, here's my > attempt at providing something we can start using and implementing > today, as a bridge to future specifications. If you'd prefer to read > it nicely formatted, you can find an HTML version in progress at: > > https://gist.github.com/pjeby/62e3892cd75257518eb0 > > I'm very interested in feedback from server and framework developers > with relevant experience to help close the "open issues and questions" > section. Questions about the content or feedback on its presentation > would also be very helpful. > > (For now, the text is in markdown, but of course I will switch it to > ReST once it begins stabilizing.) > > > # The WSGI Middleware Escape for Native Server APIs > > # Overview > > This document specifies a proposed standard WSGI extension that allows > WSGI applications to "escape" the standard WSGI API and access native > web server APIs, such as websockets, HTTP/2 features, or > Twisted/tulip-style asynchronous APIs. > > The proposed extension, the Middleware Escape for Nativer Server APIs > or "MENSA", allows WSGI to continue to be used for the 98% of typical > web application use cases that fall within the basic HTTP/1.0 > "request/response" paradigm, while allowing the 2% of use cases with > more sophisticated requirements to still benefit from "inbound" WSGI > middleware for sessions, authentication, authorization, routing, and > so forth, as well as keeping the other advantages of sharing the same > process with other WSGI code. > > Specifically, the MENSA protocol allows a WSGI application to > *dynamically* switch at runtime from using a standard WSGI response, > to using a web server's "native" API to handle the current request > (and possibly subsequent ones), subject to certain conditions. > > This approach provides present-day WSGI applications and frameworks > with a smooth upward migration path in the event that they require > access to websockets, HTTP/2-specific features, etc. With it: > > * Web servers can expose their native API to any WSGI application or > framework > It's kind of already the case with all the existing servers. They all perform the stream reading and HTTP parsing in their own native way and then adapt those to WSGI. Basically, all existing Python HTTP servers do this already. For some servers, you can even bypass the WSGI mapping altogether if you know you're only stay in the framework native-land. > > * Application developers can use existing middleware, libraries, or > frameworks to handle front-end tasks like routing and authentication > > Shouldn't we drop the middleware idea altogether? Maybe I'm being bold but when I look at some popular frameworks, many seem to escape WSGI itself already. They take a WSGI context and transform it to their native context. Then back again from their native response to the WSGI context (with a performance penalty in the process): * Django https://github.com/django/django/blob/master/django/core/handlers/wsgi.py#L83 * CherryPy https://bitbucket.org/cherrypy/cherrypy/src/4939ea9fbe6b0def376fbb349c039c38b0a8994b/cherrypy/_cpwsgi.py?at=default#cl-223 * web2py https://github.com/web2py/web2py/blob/master/gluon/globals.py#L154 Others stick closer to WSGI all the way but still hide it more or less: * Flask/Werkzeug provide properties above the environ dictionary itself https://github.com/mitsuhiko/werkzeug/blob/master/werkzeug/wrappers.py * Bottle has its own wrapper as well: https://github.com/defnull/bottle/blob/master/bottle.py#L995 * Pyramid is based on WebOb Servers seem happy enough to expose HTTP through WSGI for convenience and compatibility but, frameworks use native objects and workflow and forget about WSGI altogether. I seldom see them expose the environ or start_response at a high level. Those details are kept hidden to respect (sometimes brokenly) the WSGI protocol. -- - Sylvain http://www.defuze.org http://twitter.com/lawouach ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] WSGI and asyncio (tulip)?
Hi, 2014-10-14 18:47 GMT+02:00 Guido van Rossum : > > > I'm wondering if a small extension to the WSGI protocol might be > sufficient to support this: the special environ variable "wsgi.async_input" > could optionally be tied to a standard asyncio stream reader ( > https://docs.python.org/3/library/asyncio-stream.html#streamreader), from > which bytes can be read using "yield from stream.read([nbytes])" or "yield > from stream.readline()". > > If we support async backends by simply escaping WSGI, don't you feel it kind of make most of the whole discussion moot? To me, asyncio already provides a de-facto standard API for asynchronous backends and Tornado/Twisted provide a high level API on top of it. I have to say, I don't precisely grasp what WSGI actually wishes to bring to the table. As I said in a different thread, most frameworks seem eager to wrap the environ dictionary and hide away all of the WSGI internals (wasting CPU cycles in the process). Is there rationale for continuining down that road? -- - Sylvain http://www.defuze.org http://twitter.com/lawouach ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] WSGI and asyncio (tulip)?
2014-10-14 20:57 GMT+02:00 Justin Holmes : > To me, asyncio already provides a de-facto standard API for > asynchronous backends and Tornado/Twisted provide a high level API on > top of it. I have to say, I don't precisely grasp what WSGI actually > wishes to bring to the table. > > I guess if we're really talking about this, the lowest common > denominator is a set of cognizable abstractions for HttpRequest and > HttpResponse, right? > > That's my feeling as well. But, considering the sheer amount of content produced in other threads, I guess I'm missing the point somehow ;) -- - Sylvain http://www.defuze.org http://twitter.com/lawouach ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
