Re: [Web-SIG] Collating follow-up on the future of WSGI
because i am not speaking about making a specification, but a way to expose in the API (environ) custom extensions that a server want to experiment. there are actually no easy way except checking "wsgi." indeed but that doesn't make it as clear as a separate namespace where to put all server extensions could be. Like capability field is in imap world. Also I am not trying to force anything, I want to discuss about a possible update of the wsgi spec which I thought was this thread about. What I just want to discuss is the *current* usage of some extensions that have been rapidly dismissed as unworkable. Like I said I will come with a more formal specification about them but I wanted to discuss first about and collect counter arguments which are good too. - benoît On Thu, 21 Jan 2016 at 07:24, Graham Dumpleton wrote: > I am still confused as to why you keep talking as if you seemingly are > trying to force extensions to the existing WSGI specification into the core > WSGI specification when an alternative has already been cited. > > Extensions and the process for describing them and getting them accepted, > plus the appropriate WSGI environ prefix, as has been mentioned before, is > what is covered by: > > http://wsgi.readthedocs.org/en/latest/specifications.html > > Being an extension means it is entirely optional for a WSGI server to try > and implement it, thus allowing WSGI servers/adapters that cannot implement > something to skip them. They stand as separate documents and would never > become part of the core WGSI PEP. > > Is there an issue with doing extensions per the process, and in the WSGI > environ namespace, that was outlined in that URL? You seem to be suggesting > a completely new way of handling extensions and ignoring what was laid down > before. > > So no one is saying you can’t have extensions, and that separate process > gives you all the scope you need to do it. > > In drafting your specification just fit it reference to what is described > in that URL, using ‘x-wsgiorg.’ prefix keys. > > Graham > > > On 21 Jan 2016, at 4:13 PM, Benoit Chesneau wrote: > > I am not speaking about websockets. You could use it for SSE, or some apps > could use the Upgrade header to upgrade from http to their own protocol > etc... The only discussion i saw about websockets are about the addition of > an async api or an external api. I am not describing that. I am speaking > about providing a low level abstraction like wsgi.input but adding to it > the support of output. (I was referring to wsgi.multithread...). This low > level interface would allow anyone to provide its own > implementation(server) or usage (application) still acting as a *gateway* . > > Also who are "we"? I am starting to think the discussion is already done > and only obscure details like the content_length or headers encoding should > be discussed. The RAW_SOCKET have been added on demand of the gunicorn > users. Such thing also exist in things like cherrypy if I remember. A lot > of code around have been created over it. So before deciding it's > unworkable or whatever I strongly invite you to consider it as an addition > to the environ. And since some servers need to pass the data differently I > then suggest a Resource object on which you can read and write and > eventually poll. This is not a websocket but more a proxy ressource to the > client connexion. I will come back asap with a small spec. > > I also propose a second addition to the protocol that formalize the > addition of extensions to the protocol by the servers if they want to. > Having for example something like "`wsg.extensions` . Such addition would > help anyone to experiment changes over the wsgi before making such changes > in the specification by itself possibly. > > I think we have a good opportunity to extend the WSGI specification to > allow the users to take over the new challenges on the web without forcing > them to use a concurrency mode or skip completely the WSGI spec. The > interest I see in WSGI is its simplicity and low level interface allowing > users to build whatever they want over it. The different workers and their > support of different concurrency models and framework in gunicorn let me > think it is possible. Are the participants of this thread ready to discuss > it? > > - benoît > > On Wed, 20 Jan 2016 at 23:37, Graham Dumpleton > wrote: > >> On 21 Jan 2016, at 9:27 AM, Benoit Chesneau wrote: >> >> again. any server can do such implementation if we create a new Resource >> abstraction. This abstraction would expose a common api to read and write. >> The implementation would be specific to the server. >> >> >> If you mean not exposing the raw
Re: [Web-SIG] Collating follow-up on the future of WSGI
I am not speaking about websockets. You could use it for SSE, or some apps could use the Upgrade header to upgrade from http to their own protocol etc... The only discussion i saw about websockets are about the addition of an async api or an external api. I am not describing that. I am speaking about providing a low level abstraction like wsgi.input but adding to it the support of output. (I was referring to wsgi.multithread...). This low level interface would allow anyone to provide its own implementation(server) or usage (application) still acting as a *gateway* . Also who are "we"? I am starting to think the discussion is already done and only obscure details like the content_length or headers encoding should be discussed. The RAW_SOCKET have been added on demand of the gunicorn users. Such thing also exist in things like cherrypy if I remember. A lot of code around have been created over it. So before deciding it's unworkable or whatever I strongly invite you to consider it as an addition to the environ. And since some servers need to pass the data differently I then suggest a Resource object on which you can read and write and eventually poll. This is not a websocket but more a proxy ressource to the client connexion. I will come back asap with a small spec. I also propose a second addition to the protocol that formalize the addition of extensions to the protocol by the servers if they want to. Having for example something like "`wsg.extensions` . Such addition would help anyone to experiment changes over the wsgi before making such changes in the specification by itself possibly. I think we have a good opportunity to extend the WSGI specification to allow the users to take over the new challenges on the web without forcing them to use a concurrency mode or skip completely the WSGI spec. The interest I see in WSGI is its simplicity and low level interface allowing users to build whatever they want over it. The different workers and their support of different concurrency models and framework in gunicorn let me think it is possible. Are the participants of this thread ready to discuss it? - benoît On Wed, 20 Jan 2016 at 23:37, Graham Dumpleton wrote: > On 21 Jan 2016, at 9:27 AM, Benoit Chesneau wrote: > > again. any server can do such implementation if we create a new Resource > abstraction. This abstraction would expose a common api to read and write. > The implementation would be specific to the server. > > > If you mean not exposing the raw socket and having a separate high level > API for implementing something like WebSocket this was already talked > about. The suggestion was that it should not be a part of WSGI. Develop > that API independently with no link to WSGI. The idea of upgrading from > WSGI to a different API isn’t practical for various WSGI servers as it > isn’t possible to unwind the state of the connection path created to get to > point of handling the WSGI application. The better scenario is that the > switch to an alternate WebSocket API is handled completely within the web > server however it needs to handle it, when it needs to handle it, and not > be reliant on going into a WSGI application which then says, oh, I actually > need that to be WebSocket. > > Now like we have wsgi.thread I would instead suggest to add a system of > capability or extension like in smtp, imap, ... so the servers that > implement a specific extension can legally published it. Would it work for > you? > > > Since there is nothing in WSGI environ called wsgi.thread now I have no > idea what you are really suggesting here. > > Graham > > benoit > > On Wed, 20 Jan 2016 at 21:28, Graham Dumpleton > wrote: > >> >> On 21 Jan 2016, at 2:48 AM, Benoit Chesneau wrote: >> >> >> >> On Wed, Jan 20, 2016 at 1:57 AM Robert Collins >> wrote: >> >>> On 20 January 2016 at 12:04, Benoit Chesneau >>> wrote: >>> >>> > >>> > not at all. But I made the assumption that the wsgi server maintained a >>> > thread directly or not where the python application is running . >>> > >>> > In any case there is some sort of wrapping done in the same >>> thread/process >>> > where the python application is running. And then nothing stop to give >>> the >>> > socket away to the application and tell to the server to stop to >>> communicate >>> > with it. >>> >>> What socket? >>> >>> Data could be being passed by shm, for instance. >>> >>> -Rob >>> >>> >> While shared memory would be quite a bad idea, then why not. I still >> don't see why having a way to upgrade the connection can't be done. >> >> Call it I
[Web-SIG] Fwd: Collating follow-up on the future of WSGI
-- Forwarded message - From: Benoit Chesneau Date: Thu, 21 Jan 2016 at 06:12 Subject: Re: [Web-SIG] Collating follow-up on the future of WSGI To: Graham Dumpleton I am not speaking about websockets. You could use it for SSE, or some apps could use the Upgrade header to upgrade from http to their own protocol etc... The only discussion i saw about websockets are about the addition of an async api or an external api. I am not describing that. I am speaking about providing a low level abstraction like wsgi.input but adding to it the support of output. (I was referring to wsgi.multithread...). This low level interface would allow anyone to provide its own implementation(server) or usage (application) still acting as a *gateway* . Also who are "we"? I am starting to think the discussion is already done and only obscure details like the content_length or headers encoding should be discussed. The RAW_SOCKET have been added on demand of the gunicorn users. Such thing also exist in things like cherrypy if I remember. A lot of code around have been created over it. So before deciding it's unworkable or whatever I strongly invite you to consider it as an addition to the environ. And since some servers need to pass the data differently I then suggest a Resource object on which you can read and write and eventually poll. This is not a websocket but more a proxy ressource to the client connexion. I will come back asap with a small spec. I also propose a second addition to the protocol that formalize the addition of extensions to the protocol by the servers if they want to. Having for example something like "`wsg.extensions` . Such addition would help anyone to experiment changes over the wsgi before making such changes in the specification by itself possibly. I think we have a good opportunity to extend the WSGI specification to allow the users to take over the new challenges on the web without forcing them to use a concurrency mode or skip completely the WSGI spec. The interest I see in WSGI is its simplicity and low level interface allowing users to build whatever they want over it. The different workers and their support of different concurrency models and framework in gunicorn let me think it is possible. Are the participants of this thread ready to discuss it? - benoît On Wed, 20 Jan 2016 at 23:37, Graham Dumpleton wrote: > On 21 Jan 2016, at 9:27 AM, Benoit Chesneau wrote: > > again. any server can do such implementation if we create a new Resource > abstraction. This abstraction would expose a common api to read and write. > The implementation would be specific to the server. > > > If you mean not exposing the raw socket and having a separate high level > API for implementing something like WebSocket this was already talked > about. The suggestion was that it should not be a part of WSGI. Develop > that API independently with no link to WSGI. The idea of upgrading from > WSGI to a different API isn’t practical for various WSGI servers as it > isn’t possible to unwind the state of the connection path created to get to > point of handling the WSGI application. The better scenario is that the > switch to an alternate WebSocket API is handled completely within the web > server however it needs to handle it, when it needs to handle it, and not > be reliant on going into a WSGI application which then says, oh, I actually > need that to be WebSocket. > > Now like we have wsgi.thread I would instead suggest to add a system of > capability or extension like in smtp, imap, ... so the servers that > implement a specific extension can legally published it. Would it work for > you? > > > Since there is nothing in WSGI environ called wsgi.thread now I have no > idea what you are really suggesting here. > > Graham > > benoit > > On Wed, 20 Jan 2016 at 21:28, Graham Dumpleton > wrote: > >> >> On 21 Jan 2016, at 2:48 AM, Benoit Chesneau wrote: >> >> >> >> On Wed, Jan 20, 2016 at 1:57 AM Robert Collins >> wrote: >> >>> On 20 January 2016 at 12:04, Benoit Chesneau >>> wrote: >>> >>> > >>> > not at all. But I made the assumption that the wsgi server maintained a >>> > thread directly or not where the python application is running . >>> > >>> > In any case there is some sort of wrapping done in the same >>> thread/process >>> > where the python application is running. And then nothing stop to give >>> the >>> > socket away to the application and tell to the server to stop to >>> communicate >>> > with it. >>> >>> What socket? >>> >>> Data could be being passed by shm, for instance. >>> >>> -Rob >>> >>> >>
Re: [Web-SIG] Collating follow-up on the future of WSGI
again. any server can do such implementation if we create a new Resource abstraction. This abstraction would expose a common api to read and write. The implementation would be specific to the server. Now like we have wsgi.thread I would instead suggest to add a system of capability or extension like in smtp, imap, ... so the servers that implement a specific extension can legally published it. Would it work for you? benoit On Wed, 20 Jan 2016 at 21:28, Graham Dumpleton wrote: > > On 21 Jan 2016, at 2:48 AM, Benoit Chesneau wrote: > > > > On Wed, Jan 20, 2016 at 1:57 AM Robert Collins > wrote: > >> On 20 January 2016 at 12:04, Benoit Chesneau wrote: >> >> > >> > not at all. But I made the assumption that the wsgi server maintained a >> > thread directly or not where the python application is running . >> > >> > In any case there is some sort of wrapping done in the same >> thread/process >> > where the python application is running. And then nothing stop to give >> the >> > socket away to the application and tell to the server to stop to >> communicate >> > with it. >> >> What socket? >> >> Data could be being passed by shm, for instance. >> >> -Rob >> >> > While shared memory would be quite a bad idea, then why not. I still don't > see why having a way to upgrade the connection can't be done. > > Call it I/O resource or Socket, the issue is the same. At the end nothing > stop the server to pass the control to the app. If we forget the socket > (which is btw the simplest design) then the server could stop to control > the I/O resource when the application ask it to do it. At some point either > a garbage collection or a basic resource return/claim flow could be used to > definitely free the resource. > > The thing behind that is that it would allow the WSGI spec to only focus > on providing a strict gateway workflow without forcing the application to > adopt a concurrency model aync or not. > > > No one has said you cannot do it. because though it is only able to be > implemented in a subset of WSGI servers/adapters, then it doesn’t seem > appropriate that it be a part of the core WSGI specification. > > This is the role of a WSGI extension as found at: > > http://wsgi.readthedocs.org/en/latest/specifications.html > > So go talk to the authors of uWSGI, and the other couple of packages > available for trying to plug these into some of the pure Python based WSGI > servers and come to an agreement between yourselves as to a standard way of > doing it and the extension specification can be added to the wsgi.org > site. > > Graham > > ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Collating follow-up on the future of WSGI
On Wed, Jan 20, 2016 at 1:57 AM Robert Collins wrote: > On 20 January 2016 at 12:04, Benoit Chesneau wrote: > > > > > not at all. But I made the assumption that the wsgi server maintained a > > thread directly or not where the python application is running . > > > > In any case there is some sort of wrapping done in the same > thread/process > > where the python application is running. And then nothing stop to give > the > > socket away to the application and tell to the server to stop to > communicate > > with it. > > What socket? > > Data could be being passed by shm, for instance. > > -Rob > > While shared memory would be quite a bad idea, then why not. I still don't see why having a way to upgrade the connection can't be done. Call it I/O resource or Socket, the issue is the same. At the end nothing stop the server to pass the control to the app. If we forget the socket (which is btw the simplest design) then the server could stop to control the I/O resource when the application ask it to do it. At some point either a garbage collection or a basic resource return/claim flow could be used to definitely free the resource. The thing behind that is that it would allow the WSGI spec to only focus on providing a strict gateway workflow without forcing the application to adopt a concurrency model aync or not. - benoit. ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Collating follow-up on the future of WSGI
On Tue, Jan 19, 2016 at 11:58 PM Robert Collins wrote: > On 20 January 2016 at 05:55, Cory Benfield wrote: > > All, > > > > Thanks so much for your feedback to my original request for comments on > the future of WSGI. You provided a ton of really useful feedback: when > printed out on my printer it ended up at about 50 pages of information that > was really engaging reading. I also want to thank you all for keeping the > tone of the discussion so positive. On an topic like this one it can get > tricky and emotionally charged very easily, and you did a great job of > avoiding that problem. > > > > I spent a few hours this morning going over your feedback and trying to > extract some common threads. Altogether I believe that most participants > were mostly in agreement over the direction we should take, with a few > outliers in each case. I’d like to summarise what I believe were the big > take-aways from that discussion to confirm that I’ve understood everyone. > > > > I’d also like the members of this SIG to take this opportunity to > discuss these proposals more concretely. Rather than expressing our > sentiments about WSGI and its future more generally, I want people to > critique and offer opinions on *these specific proposals*. The goal here is > to get an understanding of whether these are worth doing, how we need to > prioritise the work, and whether there’s anything we’re missing. > > > > The below is formatted in restructured text for clarity, because it’s > quite a lot of information. > > > > > > Concrete Proposals > > == > > > > WSGI > > > > > > Overall, there was strong and fairly unanimous sentiment that WSGI > itself should more or less be left alone. Minor adjustments would be > valuable, and we should pursue them, but the contributors do not believe > that making a substantial revision to WSGI would be adviseable. > > > > I therefore propose we revise the WSGI specification to WSGI 1.1 and > then consider it "final". The following points were raised for revising > WSGI. > > Sure. > > > Asynchronous WSGI > > ~ > > > > This was generally regarded as too substantial a change to shoehorn into > WSGI. > > > > Benoit proposed that we could achieve this change by adding a correlator > to WSGI. This would allow servers to associate a given call to ``write()`` > with a specific request/response (most useful in HTTP/2 where this could > correspond to a stream ID). This would then allow WSGI to transition to a > purely callback-based model that could in principle cohabitate with an > async protocol in Python. > > Uh, WSGI doesn't need to be involved in this. Servers can already pass > arbitrary objects as write, permitting any correlation they want. I'd > want to see something very specific to consider the impact in terms of > WSGI, or WSGI-NG or whatever we do in future. > > > This proposal is worth highlighting not becuase I think we should pursue > it with our revision of WSGI, but because it's worth considering for any > future specification we come up with. Note also that Graham pointed out > that this would require some careful rethinking of reads from > ``wsgi.input``. > > > > > Server Push > > ~~~ > > > > We can support HTTP/2 server push using Link headers. This could > optionally be supplemented by defining a WSGI extension that provides a > callable for doing server push, which would be a mild improvement over the > Link header approach. A simple proposal for how to do this can easily be > implemented without revising the WSGI specification, and may not even need > to be enshrined in a PEP. However, if we revise the WSGI PEP we may want to > provide a small section that indicates how to add these headers. > > Since the point of WSGI is interop, we should document the expected > interop fashion for this, whatever it is. > > > Socket Escape Hatch > > ~~~ > > > > Aside from Benoit, server operators were unanimously dismissive of the > idea of a socket 'escape hatch'. In general it seems like servers would not > be capable of achieving this. I think, therefore, this idea is unworkable. > > I thin Benoit is making assumptions about what a 'server' is that are > not implied by WSGI - and this is leading to some confusion. Servers > are not necessarily Python processes :). > > not at all. But I made the assumption that the *wsgi* server maintained a thread directly or not where the python application is running . In any case there is some sort of wrapping done in the same thread/process where the python application is running. And then nothing stop to give the socket away to the application and tell to the server to stop to communicate with it. - benoit ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Collating follow-up on the future of WSGI
On Tue, Jan 19, 2016 at 11:34 PM Graham Dumpleton < graham.dumple...@gmail.com> wrote: > > On 20 Jan 2016, at 8:29 AM, Benoit Chesneau wrote: > > > > On Tue, Jan 19, 2016 at 10:49 PM Graham Dumpleton < > graham.dumple...@gmail.com> wrote: > >> >> On 20 Jan 2016, at 7:43 AM, Benoit Chesneau wrote: >> >> I will make a more complete answer soon. But about: >> >> >>> >>> Socket Escape Hatch >>> ~~~ >>> >>> Aside from Benoit, server operators were unanimously dismissive of the >>> idea of a socket 'escape hatch'. In general it seems like servers would not >>> be capable of achieving this. I think, therefore, this idea is unworkable. >>> >>> >> Well it does work. This is how websockets works in gunicorn. Escape is >> not the right term anyway. Think it as a socket *upgrade*. And then you >> would wonder why it would be unworkable. After all this is how SSL sockets >> works, so is the protocol negotiation in http2 ... >> >> There is nothing magic there until you try to over engineer the stuff. >> Upgrading a sockets means that you tell to the server to forget it. This is >> how most concurrent servers work today. >> >> >> The problem was that it would only work in a WSGI server where the >> original request was accepted on a socket in the same process as the WSGI >> application is running. It cannot work where where the WSGI application is >> behind a bridging protocol. >> > >> So it can’t work for CGI, SCGI, FASTCGI, mod_wsgi daemon mode and >> possibly other implementations. >> > > uh? But we don't care about bridging protocols. In WSGI (the gateway), the > server accept the socket and anyway pass it to the application via actually > a wrapper. Then expect a response from the application. > > Upgrading a socket would simply mean that the server will forget it (and > then consider its job done) once it got an appropriate response from the > application. How this is unworkable? > > > Bridging protocols such as FASTCGI do not provide an ability to upgrade > the connection end to end. > > That is, yes you could pass the raw socket to the WSGI application when > behind FASTCGI, but you are passing it a socket from same process where > data being received (and expected to be sent), is using FASTGCI message > frames. It is not a raw HTTP socket connection. > > There is no way to send a message back to the front end side of the > bridged connection where the raw HTTP socket is, to tell the client side of > the FASTCGI implementation to stop treating it as a FASTCGI connection to > backend process and then suddenly start acting as a raw socket pass through. > > But how it is related to the WSGI protocol? It is expected that the wsgi server accept on a socket (directly or behind a proxy) and provide enough to the application so they can read and write. Upgrading or escaping in WSGI/Server sense would mean that it would skip the dialog with the application once the application told it to do. For the rest, the application would do what it want on the socket, even giving it to another process/thread. - benoit ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Collating follow-up on the future of WSGI
On Tue, Jan 19, 2016 at 10:49 PM Graham Dumpleton < graham.dumple...@gmail.com> wrote: > > On 20 Jan 2016, at 7:43 AM, Benoit Chesneau wrote: > > I will make a more complete answer soon. But about: > > >> >> Socket Escape Hatch >> ~~~ >> >> Aside from Benoit, server operators were unanimously dismissive of the >> idea of a socket 'escape hatch'. In general it seems like servers would not >> be capable of achieving this. I think, therefore, this idea is unworkable. >> >> > Well it does work. This is how websockets works in gunicorn. Escape is > not the right term anyway. Think it as a socket *upgrade*. And then you > would wonder why it would be unworkable. After all this is how SSL sockets > works, so is the protocol negotiation in http2 ... > > There is nothing magic there until you try to over engineer the stuff. > Upgrading a sockets means that you tell to the server to forget it. This is > how most concurrent servers work today. > > > The problem was that it would only work in a WSGI server where the > original request was accepted on a socket in the same process as the WSGI > application is running. It cannot work where where the WSGI application is > behind a bridging protocol. > > So it can’t work for CGI, SCGI, FASTCGI, mod_wsgi daemon mode and possibly > other implementations. > uh? But we don't care about bridging protocols. In WSGI (the gateway), the server accept the socket and anyway pass it to the application via actually a wrapper. Then expect a response from the application. Upgrading a socket would simply mean that the server will forget it (and then consider its job done) once it got an appropriate response from the application. How this is unworkable? - benoit ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Collating follow-up on the future of WSGI
I will make a more complete answer soon. But about: > > Socket Escape Hatch > ~~~ > > Aside from Benoit, server operators were unanimously dismissive of the > idea of a socket 'escape hatch'. In general it seems like servers would not > be capable of achieving this. I think, therefore, this idea is unworkable. > > Well it does work. This is how websockets works in gunicorn. Escape is not the right term anyway. Think it as a socket *upgrade*. And then you would wonder why it would be unworkable. After all this is how SSL sockets works, so is the protocol negotiation in http2 ... There is nothing magic there until you try to over engineer the stuff. Upgrading a sockets means that you tell to the server to forget it. This is how most concurrent servers work today. - benoit ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] WSGI 2.0 Round 2: requirements and call for interest
what do you need asynchronous? And how the current callback system can't fit the needs of an an asynchronous lib? what do you miss actually? Note that http and http2 are not asynchronous. Imo we need a new WSGI spec and a Messaging gateway spec. but these are orthogonal discussions imo. - benoit On Wed, 6 Jan 2016 at 12:52, Amber "Hawkie" Brown wrote: > > > On 4 Jan 2016, at 20:27, Cory Benfield wrote: > > > > All, > > > > **TL;DR: What do you believe WSGI 2.0 should and should not do? Should > we do it at all?** > > > > It’s a new year, and that means it’s time for another attempt to get > WSGI 2.0 off the ground. Many of you may remember that we attempted to do > this last year with Rob Collins leading the charge, but unfortunately > personal commitments made it impossible for Rob to keep pushing that > attempt forward. > > > > Since then, the need for a revision of WSGI has become even more > apparent. Casual discussion on the web has indicated that application > developers are uncomfortable with the limitations of WSGI. These > limitations are providing an incentive for both application developers and > server developers to take an end-run around WSGI in an attempt to get a > framework that is more suitable for the modern web. A great example of the > result of WSGI’s deficiencies is Andrew Godwin’s channels work[0] for > Django, which represents a paradigm shift in application development that > takes it far away from what WSGI is today. > > > > For this reason, I think we need to try again to get WSGI 2.0 off the > ground. But I don’t believe we can do this without getting broad consensus > from the developer community that a revision to WSGI is needed, and without > understanding what developers need from a new revision of WSGI. This should > take into account the prior discussions we’d had on this thread: however, > I’m also going to actively solicit feedback from some of the more notable > WSGI implementers, to ensure that whatever comes out of this SIG is > something that they would actually use. > > > > This WG already had a list of requirements, which are as follows: > > > > - Support servers speaking HTTP/1.x, HTTP/2 and Websockets (potentially > all on a single port). > > - Support graceful degradation for applications that can use HTTP/2 but > still support HTTP/1.x requests. > > - Graceful incremental adoption path - no upgrade-all-components > requirement baked into the design. > > - Support Python 2.7 and 3.x (where x is not yet discussed) > > - Support the existing ecosystem of containers (such as mod_wsgi) with > the new API. We want a clean, fast and approachable API, and we want to > ensure that its no less friendly to work with than WSGI, for all that it > will expose much more functionality. > > - Apps need to be able to tell what protocol is in use, and what > optional features are available. For instance, HTTP/2 PUSH PROMISE is an > optional feature that can be disabled by clients. Websockets needs to > expose a socket like object, and so on. > > - Support websockets > > - Support HTTP/2 > > - Support HTTP/1.x (which may be just 'point at PEP-’.) > > - Continue to support lightweight shims being built on top such as > https://github.com/Pylons/webob/blob/master/webob/request.py > > > > I believe that all of these requirements are up for grabs, and subject > to change and consensus discussion. In this thread, then, I’d like to hear > from people about these requirements and others. What do you believe WSGI > 2.0 should do? Just as importantly, what do you believe it should not do? > What prior art should we take into account? Should we bother revising WSGI > at all, or should we let the wider application ecosystem pursue its own > solutions à la Django's channels? Should we simply adopt Andrew Godwin’s > ASGI draft[1] on which channels is based and call *that* WSGI 2.0? > > > > Right now I want this to be very open. I’d like people to come up with a > broad statement listing what they believe should and should not be present > in WSGI. This first stage of the work is very general: I just want to get a > feeling for what the community believes is important. Once we’re done with > that, if the consensus is that this work is worth pursuing, I’ll come up > with an initial draft that we can start making concrete changes to. > > > > In the short term, I’m going to keep this consultation open for **at > least two weeks**: that is, I will not start working on an initial draft > PEP until at least the **18th of January**. If you believe there are > application or server developers that should be involved in this > discussion, please reach out to them and point them to this list. I > personally have CC’d some people that I believe need to be involved in the > discussion, but please reach out to others as well. > > > > I’d really love to come to the end of 2016 with a solid direction for > the future of web programming in Python. I’m looking forward to working > with you all on achi
Re: [Web-SIG] WSGI 2.0 Round 2: requirements and call for interest
On Wed, Jan 6, 2016 at 10:29 AM Graham Dumpleton wrote: > > On 6 Jan 2016, at 12:13 AM, Benoit Chesneau wrote: > > So for me what should be WSGI 2? WSGI 2 should add against WSGI 1 the > following: > > - tell to the application it is actually an HTTP2 request (maybe > populating a wsgi.http2 true env) > > > In CGI implementations you would for HTTP/1.1 already get: > > SERVER_PROTOCOL: 'HTTP/1.1’ > > Under HTTP/2 when I tested some time back, I recollect it came through as > one would assume would be expected: > > SERVER_PROTOCOL: ‘HTTP/2’ > > Is there any reason that this existing CGI variable wouldn’t be sufficient > for this purpose? > > Graham > True. I don't think there is a need for another env. The current way also work for the future :) - benoît ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] WSGI 2.0 Round 2: requirements and call for interest
On Wed, Jan 6, 2016 at 10:19 AM Aymeric Augustin < aymeric.augus...@polytechnique.org> wrote: > Hello Benoît, > > Thanks for clarifying that you also had the reverse problem in mind, > headers sent by applications. This side is less problematic in the sense > that application authors can adapt to stronger requirements. > > In general this is a bit of a mess due to differences between what the RFC > 2616 says and what browsers do in practice. That’s why I believe the > pragmatic solution is to exchange bytes. (This isn’t a major issue in the > grand scheme of things anyway.) > > Best regards, > > -- > Aymeric. > > Since sockets are only accepting bytes in python 3.x, having everything as bytes would also allow the server to process the response more easily... - benoît ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] WSGI 2.0 Round 2: requirements and call for interest
On Tue, Jan 5, 2016 at 3:17 PM Aymeric Augustin < aymeric.augustin.2...@polytechnique.org> wrote: > Hello Benoît, > > > Le mardi 5 janvier 2016 14:13:48 UTC+1, Benoit Chesneau a écrit : >> >> Header formats which are btw US-ASCII in the HTTP spec now, could be >> already solved if only the frameworks could comply with the spec instead of >> trying to impose their own rules. >> > > That's just a detail, but either I misunderstood you or you blamed the > wrong side here. > > Non-ASCII data in request headers isn't a problem created by frameworks, > it's a problem created by (possibly non compliant) user-agents. > I had in mind this ticket: https://github.com/benoitc/gunicorn/issues/1151 As of today, because some applications are still sending response in a a non compliant way we are trying to recode the headers on the server side so we can send them. Today like in apache 2 (and I think nginx) we now just ignore headers that can't be encoded in us-ascii. If all applications/framework would give us the headers as Latin1 it wouldn't be a major problem, but that's not the case. > > If future-WSGI guaranteed that HTTP header values provided in environ only > contain ASCI, fameworks would be happy. Servers would likely have to > respond 400 to requests containing non-ASCII headers, which would likely be > considered a problematic backwards-incompatibility. It would go against the > IETF principle of being tolerant in what a system accepts. > We should also update the spec to reflect the latest changes in the HTTP specs to force applications to send to the gateway US-ASCII headers. > > If future-WSGI provided header values as bytes, frameworks would be happy > as well. That would be my preference, because the application is in the > best position to pick a charset for decoding the values (that would be > UTF-8 in general). > > If future-WSGI insists on decoding header values with an arbitrary > encoding, I believe it should do so with UTF-8 rather than ISO-8859-1. "The > server is decoding with ISO-8859-1 so the application can reencode to get > the raw bytes" never sounded like a compelling argument to me. It will > still be wrong in theory, but it will generally give the right results in > practice. > Hmm but, actually the HTTP spec insist that headers are neither utf-8, neither latin1 (iso8859-1) but US-ASCII: https://github.com/benoitc/gunicorn/issues/1151#issuecomment-158884740 so native strings or bytes are fine for me until we make sure that we are sending and receiving US-ASCII. > Best regards, > > > -- > Aymeric. > > PS: if you find Django trying to impose its own rules, I'll do my best to > correct that. As much as I can speak for the Django team, this isn't our > intent. Please flag such cases so we can make sure there's no > misunderstanding. Thanks! > > Thanks! I will if needed :) - benoît ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] WSGI 2.0 Round 2: requirements and call for interest
Hi all, Hopefully this discussion won't turn in another useless political discussion :) About the need of a new spec aka WSGI 2 or whatever the name you want to I would say it's definitely needed. But contrary to the others I don't think it has to be that new, or breaking. If you follow closely the way HTTP2 has been designed it can be considered as an extension of HTTP 1.x, keeping the API fixing the following the connections latency and server resource usages. Grosso modo, the specifications can be split in 3 keys point - binary protocol - connections multiplexing, handling more than one 1 request / sockets - server PUSH to somehow "pre-send" data on request to reduce the latency Header formats which are btw US-ASCII in the HTTP spec now, could be already solved if only the frameworks could comply with the spec instead of trying to impose their own rules. Websockets or async things are outside the spec. All the points above are server side. Behind that the application is actually doing the same stateless request/response path. Framing/Decoding is the responsability of the server and should be hidden to the application. The only addition on the application side is the possibility to push. So for me what should be WSGI 2? WSGI 2 should add against WSGI 1 the following: - tell to the application it is actually an HTTP2 request (maybe populating a wsgi.http2 true env) - tell to the application the request id it is actually answering when receiving the data (it will be the the responsability of the application to handle it, asynchronously or not, using asyncio or not, greenlets, threads, whatever). The server by itself will only process callbacks like it is now. - expose an API to handle the PUSH api . mostly a way to buffer it on the server so it can be sent ASAP to the remote client Besides that I think WSGI 2 is missing the following: - an UPGRADE api, allowing an application to actually take the client socket control and do anything with it. In gunicorn it is handled by providing the raw socket in the environ and tell to gunicorn it can now ignore this socket: https://github.com/benoitc/gunicorn/blob/master/examples/websocket/websocket.py#L115 This is useful if an app wants to handle websockets, provides a W3C PUSH api, ... - a clear standard to provide to the application the raw URI. Again Gunicorn is providing it via the environnement using the `RAW_URI` setting. - a better way to tell the application it is actually responding (or want to answer) with chunked encoding data or stream data. Note that actually there is nothing stopping a WSGI server to handle them. Gunicorn is actually streaming any data and decode chunks on the fly. Also it is encoding for the application if the header is present. Using the FileWrapper api it works quite well. Maybe though we could improve it by providing a Stream object to distinct it from a File. So for example, the server don't have to check if it can use sendfile over it or not, which proved to be annoying across the different python version (also not efficient). It could also be used to add on server side some stuff like zero-copy and so on. Also I think a new spec must make it clear that the server should not have to be aware about the way the application is handling the request and respond to them ie asynchronously or not, using threads or not. The protocol/api should be stateless. And the application must maintains it own state without any expectation on the way the server maintain it. By that I mean that lot of people are surprised to not be able to share a pool between all the requests in gunicorn because they don't use a thread-safe (and process safe) pool. This should be part of the expectation given by the spec imo. Maybe as an extra, we should define more clearly what is a WSGI middleware. Any feedback is welcome. - benoît. On Mon, Jan 4, 2016 at 1:27 PM Cory Benfield wrote: > All, > > **TL;DR: What do you believe WSGI 2.0 should and should not do? Should we > do it at all?** > > It’s a new year, and that means it’s time for another attempt to get WSGI > 2.0 off the ground. Many of you may remember that we attempted to do this > last year with Rob Collins leading the charge, but unfortunately personal > commitments made it impossible for Rob to keep pushing that attempt forward. > > Since then, the need for a revision of WSGI has become even more apparent. > Casual discussion on the web has indicated that application developers are > uncomfortable with the limitations of WSGI. These limitations are providing > an incentive for both application developers and server developers to take > an end-run around WSGI in an attempt to get a framework that is more > suitable for the modern web. A great example of the result of WSGI’s > deficiencies is Andrew Godwin’s channels work[0] for Django, which > represents a paradigm shift in application development that takes it far > away from what WSGI is today. > > For this reason, I think we need t
Re: [Web-SIG] Draft 2: WSGI Response Upgrade Bridging
On Tuesday, October 14, 2014, Robert Collins wrote: On 14 October 2014 01:18, Benoit Chesneau wrote: >> C - Support for chunked uploads, comet, bosh and websockets is >> effectively impossible within WSGI - one ends up writing server >> specific code, and being tied to a single server - even though >> multiple servers support (some of) those things. This defeats the >> point of WSGI IMNSHO: its not that WSGI is broken or anything, its >> just that we're once again writing all our generic middleware in >> server-specific fashions. Because the world has moved on and we >> haven't. > > > Chunkedn upload is possible and already handled with Gunicorn. But there is > no standard for that. Right. Thus we need one. > For C I would separate it from the rest. This a different discussion and imo > not everything can be achieved at the same time. Maybe we should start first > by fixing them, then go for the next step anyway. So the transition could be > incremental in servers and frameworks and actually fix the current spec. What makes C a different discussion? > > For A (And C), i think we should keep the new specification enough agnostic. > Especially since HTTP 2 is not yet completely out. HTTP/2 is in last call stage: it will be entirely finished by the time we get through whatever process we have here. What do you want to see changed in the process I'm following? -Rob I meant there are 2 separate problems: fixing the current spec, and extend it **if** needed to handle the new web patterns. I am speaking more about patterns than protocols. Protocol is one thing to take in consideration of course, but actually if we redefining a spec to build (server) and interact with (apps) a gateway for the web it will be more than simply handling HTTP 2 and soon HTTP 2.1 or 3 depending on the discussions. On the server started I started recently a redesign of the core of Gunicorn to prepare it tfor this pattern and beeing able to handle the different challengenns they gives. I actually identified some common features and distinct features. In term of patterns we have the following: - start a response: detect the protocol, eventually *upgrade* it to a new protocol (HTTP -> Websockets or HTTP2 -> HTTP 1.1) - send/receive headers - PUSH pattern: HTTP 2, new PUSH specifcations from W3C, SSE - ASYNC pattern: HTTP 2 and Websockets, receiving and sending can happen at different time and on the long term - Continuous connections: keepalive, HTTP 2 channels, websockets, SSE (how do we keep/identify connections states) - Streaams: chunked encoding, http 2 channels, ... We should have a clear way to notify the application about it. Also the spec/the server should have a standard way to handle the different async frameworks not based on threads like gevent, eventlet. The application should also be able to tell to the server that an operation will take a long time and let the server take appropriate actions. Some kind of reply/no_reply pattern allowing to answer later and switch to an async pattern on the fly if the server support it. It would allows the servers to handle more gracefully some common issues like long queries when the application knows about them. Since there is a concensus on using the github tracker, should i open a tickets for these different things? Or one generic? Let me know. - benoit -- Robert Collins Distinguished Technologist HP Converged Cloud ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Draft 2: WSGI Response Upgrade Bridging
On Tuesday, October 14, 2014, Tres Seaver wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 10/13/2014 12:12 AM, Benoit Chesneau wrote: > > So I should probably know you, but I can't recollect right now what > > you do or write. > > Seriously? On *this* sig? PJE was the author of PEP 333, defining the > WSGI 1.0 spec. > > Actually i was more annoyed by the way the discussion was handled than the rest. If we can release somerhing more adapted to the current challenge and not finish like the pep this time that would be awesome. -- Sent from my Mobile ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] REMOTE_ADDR and proxys
On Sun, Oct 12, 2014 at 11:38 PM, Robert Collins wrote: > On 30 September 2014 11:47, Alan Kennedy wrote: > > > [Robert] > >> So it sounds like it should be the responsibility of a middleware to > >> renormalize the environment? > > > > In order for that to be the case, you have strictly define what > > "normalization" means. > > For a given deployment its well defined. I agree that in general its not. > > > I believe that it is not possible to fully specify "normalization", and > that > > any attempt to do so is futile. > > > > If you want to attempt it for the specific scenarios that your particular > > application has to deal with, then by all means code your version of > > "normalization" into your application. Or write some middleware to do it. > > > > But trying to make "normalization" a part of a WSGI-style specification > is > > impossible. > > I don't recall proposing that it should be in a WSGI-style spec. > > -Rob > > -- > Robert Collins > Distinguished Technologist > HP Converged Cloud > ___ > Web-SIG mailing list > Web-SIG@python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: > https://mail.python.org/mailman/options/web-sig/bchesneau%40gmail.com > All this issue looks like the problem raised (and not yet solved) recently in Gunicorn when the REMOTE_ADDR has been handled more strictly and we removed all the X-Forward-* headers handling: https://github.com/benoitc/gunicorn/issues/797 There is another case to take in consideration, when your server is answering on unix sockets, so you don't have any TCP address to present. For now we answer with an empty field. Also some application frameworks recently removed the middleware handling X-Forward-* headers. I wonder why. There is an RFC for forward headers: http://tools.ietf.org/html/rfc7239 . For me instead of trying to change the strict behaviour of REMOTE_ADDR I wonder if we shouldn't rather add a new field to the environ. Thoughts? - benoit ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Draft 2: WSGI Response Upgrade Bridging
On Mon, Oct 13, 2014 at 11:52 AM, Robert Collins wrote: > On 13 October 2014 17:12, Benoit Chesneau wrote: > > > ... > > > > > > OK, > > > > So I should probably know you, but I can't recollect right now what you > do > > or write. > > Its not clear to me who you were replying to. > I answered at the bottom of the thread so to PJE. > > Anyway I find it really disturbing the way you're actually acting > > and try to push your ideas based on private feedback coming from unknown > or > > choosing who should be a reference. That certainly not the right way to > have > > all actors on the table. Because if we go for a new WSGI spec, you > certainly > > want it. And I am speaking as one of these actors. > > As I said when folk talked about going private in the first thread on > this, I'm willing to discuss anything publically or privately - I > can't tell folk where they will be comfortable discussing things. But > I'm going to do *my* work on this in public, because I think that is > essential to get broad consensus. > > > In my opinion, if we want to go further we should first define what are > the > > problem we want to solve, and then get the feedback from all the actors > > around: > > I think I've been fairly clear about the problem *I* want to solve. > > """ > We want to create a clean common API for applications and middleware > written in a post HTTP/2 world - where single servers may accept up to > all three of HTTP/1.x, HTTP/2 and Websocket connections, and > applications and middleware want to be able to take advantage of > HTTP/2 and websockets when available, but also degrade gracefully. We > also want to ensure that there is a graceful incremental path to > adoption of the new API, including Python 2.7 support, and shims to > enable existing WSGI apps/middleware/servers to respectively be > contained, contain-or-be-contained and contain, things written to this > new API. We want a clean, fast and approachable API, and we want to > ensure that its no less friendly to work with than WSGI, for all that > it will expose much more functionality. > """ > By which problem we need to solve, I mean we need to identify clearly what are the problem not solved by the current spec. And see why, and how it is actually solved in the python world. we need to clearly identify these issues and make sure we have a comprehensive view of them. > > > - framweorks authors > > I reached out to a number of such authors directly. I encourage you to > do the same. > I could do that eventually if we are all agree on the process. > > > - libraries author > > Ditto and > > > - server authors > > Ditto :). > > > If you don't have all actors around and majors are missing, there is > > probably no point to continue. I do think the idea of having a > repository to > > collect it with people arbitrating the discussions on them on the > mailing is > > a good way to go further. Now I think we are still missing of a clear > > definition of the problem. This is from what we should start instead of > > starting by giving our philosophy on how the problem should be solved. > > Here's my definition of some of the problems: > A - there is no common spec equivalent to WSGI that permits writing > server side code that takes advantage of HTTP/2. There's *a* http/2 > server out there which one can write code for, but that code is either > specific to that servers plumbing, or plain WSGI and misses the HTTP/2 > goodness. > B - WSGI has some oddness and overheads due in large part to the > situation it was aiming to fix (which it broadly did) that perhaps we > can now come together to fix. > C - Support for chunked uploads, comet, bosh and websockets is > effectively impossible within WSGI - one ends up writing server > specific code, and being tied to a single server - even though > multiple servers support (some of) those things. This defeats the > point of WSGI IMNSHO: its not that WSGI is broken or anything, its > just that we're once again writing all our generic middleware in > server-specific fashions. Because the world has moved on and we > haven't. > Chunkedn upload is possible and already handled with Gunicorn. But there is no standard for that. For C I would separate it from the rest. This a different discussion and imo not everything can be achieved at the same time. Maybe we should start first by fixing them, then go for the next step anyway. So the transition could be incremental in servers and frameworks and actually fix the current spec. For A (And C), i think we should keep the new specification enough agnostic. Especially since HTTP 2 is not yet completely out. - benoit I ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Draft 2: WSGI Response Upgrade Bridging
On Fri, Oct 10, 2014 at 9:10 PM, PJ Eby wrote: > On Fri, Oct 10, 2014 at 8:56 AM, Graham Dumpleton > wrote: > > So PJE, please step back and do not go rushing out to create a PEP. That > is > > the worst thing you could do at this point and will only serve to deter > > people from the community contributing and so stifle proper discussion > about > > this whole topic. > > Huh? Have you *read* the PEP? The entire point of it is to provide a > basis for *experimenting* with new standards, not to "stifle > discussion" of them. It's not even an *API*, for heavens' sake. It's > just a description of how to upgrade to new standards from within > existing WSGI frameworks, without needing to tunnel responses and > without breaking subrequest middleware. > > IOW, it's a WSGI *1.0* server extension protocol, and a fairly *minor* > one at that. (Indeed, it's little more than an enhanced variation of > wsgi.file_wrapper!) It's not any sort of competitor or alternative to > what Robert's working on; it's a *stepping stone* for what Robert's > working on. > > In early discussion with Robert -- both here and on github -- it > became apparent to me that restricting post-WSGI specifications to > what can be achieved in WSGI 1 tunneling is a bad idea. So I've > created a *bridging* specification, that allows post-WSGI APIs to be > accessed from within WSGI-based apps and frameworks. > > That's *all*. > > All of the things you've mentioned as being in scope for discussion, > are *still* in scope for discussion. All *this* proposal does is show > how those things could be *accessed*, *today*, from inside existing > web apps and frameworks, once those new APIs exist. > > > > You have no more experience or mandate to be specifying a > > standard for this than anyone else. > > If by "this" you're referring to HTTP/2 or some other new post-WSGI > API, then I agree with you. But that's not what the PEP is about. > > > > By creating a PEP though that gets > > perceived by many as meaning the discussion is over. This is exactly what > > you did for PEP and which caused previous discussion about improving > > WSGI to get shutdown. > > That's an interesting perspective, but I don't see how it can be > reconciled with the facts. > > First off, I didn't write a new PEP; I wrote up some of *your* > proposed clarifications for Python 3 as WSGI 1.0.1, which was intended > to add new clarifying text to PEP 333, *not* to create a new PEP. It > was *Guido* who said it must be a new PEP, as you will see here: > > https://mail.python.org/pipermail/web-sig/2010-September/004691.html > > and here (where he even says, "Don't see this as a new spec. See it as > a procedural issue."): > > https://mail.python.org/pipermail/web-sig/2010-September/004694.html > > Second, I didn't make anybody stop discussing alternatives for moving > things forward. Nobody *ever* said to stop working on a version 2 or > even 1.1, certainly not me. See for example, this message, where I > agreed with Ian's POV that there was room for both PEP 333 fixes *and* > continued work on PEP 444: > > https://mail.python.org/pipermail/web-sig/2010-September/004662.html > > Third, and finally, as far as I can tell from the record of the > discussion back then, it was you -- and *only* you -- who suggested > that the acceptance of PEP meant the discussion was *over*. > Indeed, on your blog you actually pushed back at Alice for bringing up > more PEP 444 discussion! > > Nonetheless, discussion of PEP 444 and async APIs and such proceeded > well past the introduction of PEP , even without its original > authors' participation. And, ironically enough, your posts show up in > that discussion, bemoaning that Alice (the new PEP 444 champion) was > creating confusion by calling that proposal WSGI 2.0! > > > > The result was that the only thing that really got > > addressed in PEP was Python 3 compatibility and a lot of the other > bits > > of the WSGI specification which are poorly defined, contradictory or > > restrictive and which cause WSGI server and application developers pain > > never got addressed. If that prior discussion hadn't been shutdown in > that > > way, we could have been using a better defined and improved WSGI years > ago > > already. > > Those things didn't get addressed because *you* didn't take up the > lead -- a lead which I more than once mentioned you should take up. > For example, as I said in > https://mail.python.org/pipermail/web-sig/2010-September/004693.html : > > > The full list of things Graham and others have asked for or > > recommended would indeed require a 1.1 version at minimum, and thus a > > new PEP. But I really don't want to start down that road right now, > > and therefore hope that I can talk Graham or some other poor soul > > into shepherding a 1.1 PEP instead. ;-) > > You didn't, and haven't, taken up that slack. What you've > consistently done is mutter and grumble on the sidelines about how > it's
Re: [Web-SIG] WSGI2: write callable?
On Fri, Sep 26, 2014 at 9:58 PM, PJ Eby wrote: > On Thu, Sep 25, 2014 at 11:32 PM, Robert Collins > wrote: > > So I propose we drop the write callable, and include a queue based > > implementation in the adapter for PEP- code. > > If you're dropping write(), then you might as well drop > start_response() altogether, and replace it with returning a (status, > headers, body-iterator) tuple, as in wsgi_lite ( > https://github.com/pjeby/wsgi_lite ) or as found in other languages' > versions of WSGI. (start_response+write was only ever needed in order > to support legacy apps, so other languages never bothered.) > > wsgi_lite has a couple of other protocol extensions, namely the > 'wsgi_lite.closing' environment key, flagging callables' supported > WSGI version (for transparent interop), and the argument binding > protocol, but for the most part these are orthogonal to the calling > schema. I would suggest, however, that the calling protocol be > flagged in some way to allow easier interop. > I quite like the idea of always returning an iterator for the body it would simplify the code a lot... About returning the status and other thing, I quite agree, but imo we also need to return an extra parameter where the application or the middleware could maintain a state or something like it. Thoughts? - benoit > ___ > Web-SIG mailing list > Web-SIG@python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: > https://mail.python.org/mailman/options/web-sig/bchesneau%40gmail.com > ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] WSGI2: write callable?
On Fri, Sep 26, 2014 at 5:32 AM, Robert Collins wrote: > Is the write callable still needed? Its documented as a undesirable > thunk in PEP-; is there a good reason to keep it, or can we make > start_response return None and require the use of a generator to > supply content for the body? > > (Remembering that for backwards compatibility we're going to write an > adapter, and a generator adapter is straightforward (if tedious) using > a threading.Queue). > > I haven't done a survey, but I don't recall seeing anything except > bespoke WSGI code that used the write interface - all the frameworks > I've seen in some time use the iterator protocol. > > So I propose we drop the write callable, and include a queue based > implementation in the adapter for PEP- code. > > -Rob > > What would be the advantage of using a queue compared to simply write to the server? Internally the server can use queue, but why the client should know it? What is the reasoning behind it? - benoit > -- > Robert Collins > Distinguished Technologist > HP Converged Cloud > ___ > Web-SIG mailing list > Web-SIG@python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: > https://mail.python.org/mailman/options/web-sig/bchesneau%40gmail.com > ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] WSGI for HTTP/2.0 ?
got an idea. What about having a page collecting feedback from anyone in the python community about this topic. So we can have true data from different perspectives: developer, library/framework author, server author. I'm OK to collect the data from it and make a summary of it once it's done. The form it could take should be discussed first but imo that a good way to engage the community. What do you think? On Sat, Sep 20, 2014 at 4:17 PM, Benoit Chesneau wrote: > > > On Sat, Sep 20, 2014 at 3:22 PM, Dirkjan Ochtman > wrote: > >> On Sat, Sep 20, 2014 at 9:23 AM, Robert Collins >> wrote: >> > Well, thats certainly a challenge :). Whats the governance model here? >> > Is a PEP appropriate, and if so - that gives us a BFDL or BFDL >> > PEP-delegate to decide between bikeshed issues; and if its not a >> > bikeshed issue then resolving it is actually necessary. >> >> Yes, I think a good way forward would be to have a small cabal write a >> PEP and then announce it here for further feedback and then >> pronouncement by a BDFL or -delegate. If you want to be lead/editor, >> that sounds great. It also seems like you should definitely involve >> Graham and give credence to his thoughts. >> >> I'd be excited about this and happy to give feedback a little later >> once you've got some initial draft, as someone who likes to implement >> his applications directly on top of WSGI for now (but I've also >> implemented a couple of WebSocket servers). >> >> Cheers, >> >> Dirkjan >> >> > Last time was more about everyone wanted to discuss about the changes with > its own requirement list. Which is already conflicting. Discussing if it > should be done outside the media designed for it is already out of topic > imo. So I won't discuss about it further. > > Instead I wonder what is the appropriate medium to collect requirements > and others stuffs about it. Wiki ? Anything else? > > For a start I see these different topics > > > 1) HTTP 1.1 vs HTTP 2: > > - HTTP 1.1 and HTTP2 have quite the same high level syntax (methods, uri, > headers, ...) but the way the data is transported differs. (data are sent > by frames in HTTP 2). > - in HTTP 2, data can be encrypted and compressed. > - in HTTP2 data can pushed from the server to the clients. More data can > be sent to the client > - in HTTP2 streams are multiplexed We have the concept of data channels > and these are more like message passings. Multiplexing existed in HTTP 1.1 > with pipelines but is barely supported right now by WSGI servers. > > > The concept of data channels and the PUSH features will requires more > concurrency at the server level. > > At the application, things doesn't change that much. Everything can appear > like before. The only change is the PUSH feature. > > > 2) Websockets, SSE and other similar protocoles are completely > asynchronous. All this part is not really handled by WSGI. The way it is > generally implemented right now is awkward. The server generally extend the > WSGI protocol so the application get the socket. Then a specific library > handle the rest. > > > I actually wonder if websockets or other asynchronous protocols should be > handled by the new WSGI SPEC. Shouldn't we just standardize the way the > socket is given to another library? > > > Anyway I think we should collect all requirements at application and > server level and then start to confront the current WSGI spec to them. And > iterate. Thoughts? Any other topic? > > - benoit > > > > > > > > > ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] WSGI for HTTP/2.0 ?
On Sat, Sep 20, 2014 at 3:22 PM, Dirkjan Ochtman wrote: > On Sat, Sep 20, 2014 at 9:23 AM, Robert Collins > wrote: > > Well, thats certainly a challenge :). Whats the governance model here? > > Is a PEP appropriate, and if so - that gives us a BFDL or BFDL > > PEP-delegate to decide between bikeshed issues; and if its not a > > bikeshed issue then resolving it is actually necessary. > > Yes, I think a good way forward would be to have a small cabal write a > PEP and then announce it here for further feedback and then > pronouncement by a BDFL or -delegate. If you want to be lead/editor, > that sounds great. It also seems like you should definitely involve > Graham and give credence to his thoughts. > > I'd be excited about this and happy to give feedback a little later > once you've got some initial draft, as someone who likes to implement > his applications directly on top of WSGI for now (but I've also > implemented a couple of WebSocket servers). > > Cheers, > > Dirkjan > > Last time was more about everyone wanted to discuss about the changes with its own requirement list. Which is already conflicting. Discussing if it should be done outside the media designed for it is already out of topic imo. So I won't discuss about it further. Instead I wonder what is the appropriate medium to collect requirements and others stuffs about it. Wiki ? Anything else? For a start I see these different topics 1) HTTP 1.1 vs HTTP 2: - HTTP 1.1 and HTTP2 have quite the same high level syntax (methods, uri, headers, ...) but the way the data is transported differs. (data are sent by frames in HTTP 2). - in HTTP 2, data can be encrypted and compressed. - in HTTP2 data can pushed from the server to the clients. More data can be sent to the client - in HTTP2 streams are multiplexed We have the concept of data channels and these are more like message passings. Multiplexing existed in HTTP 1.1 with pipelines but is barely supported right now by WSGI servers. The concept of data channels and the PUSH features will requires more concurrency at the server level. At the application, things doesn't change that much. Everything can appear like before. The only change is the PUSH feature. 2) Websockets, SSE and other similar protocoles are completely asynchronous. All this part is not really handled by WSGI. The way it is generally implemented right now is awkward. The server generally extend the WSGI protocol so the application get the socket. Then a specific library handle the rest. I actually wonder if websockets or other asynchronous protocols should be handled by the new WSGI SPEC. Shouldn't we just standardize the way the socket is given to another library? Anyway I think we should collect all requirements at application and server level and then start to confront the current WSGI spec to them. And iterate. Thoughts? Any other topic? - benoit ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] WSGI for HTTP/2.0 ?
Hi, I would prefer to have this work being done transparently. If we do it rationally it could work imo. Anyway before thinking to change the protocol or criticizing it maybe we could first collect the requirements in HTTP 2 (stream and such) so we can think about possible implementations. And see what it misses in WSGI. I am thinking we could adopt the same path used to decided to go for HTTP 1.x or HTTP 2 on the client part. Ie keeping WSGI and PEP for HTTP 1.1 applications and go for a new interface in HTTP2. But such decision should be done once we have a clear view of what requires HTTP 2 and how it can be handled on the python side. Thoughts? - benoit On Sat, Sep 20, 2014 at 8:31 AM, Graham Dumpleton < graham.dumple...@gmail.com> wrote: > > On 20/09/2014, at 3:49 PM, Roberto De Ioris wrote: > > > I can help a bit (i am the uWSGI lead developer and a nginx and Cherokee > > contributor, and i have already implemented a spdy3 server last year) > > > > I honestly think that WSGI by itself needs a complete rewrite/rethink to > > be adapted to modern (ok someone could say 'fashioned') patterns (that > are > > somewhat more 'urgent' than HTTP/2), but i agree that starting thinking > > about HTTP/2 could be a good thing. > > I agree. > > Overhauling WSGI has more relevance because an underlying web server > updating itself to support HTTP 2.0 will in the main have little relevance > at the application layer as the web server is more than likely to have an > adapter layer which makes things look the same to existing modules/protocol > adapters. > > In other words, Apache adding support for HTTP 2.0 isn't going to result > in some sort of wholesale change of the Apache module interface, it would > stay the same say whether HTTP 2.0 is used, especially just as an alternate > way of doing the same thing as HTTP 1.1. In that respect, since no HTTP 2.0 > specific functionality is going to be made visible through exist > interfaces, then Apache modules or adapters for FASTCGI/SCGI etc or even > mod_wsgi are simply not going to change. > > So, overhaul WSGI as the primary aim, but within that factor in things to > allow for HTTP 2.0 functionality. > > The problem with trying to overhaul WSGI is that if it is done in an open > forum like the Web-SIG it will die of a thousand cuts, as past efforts to > update it in even minor ways have suffered. > > The only way that WSGI itself will ever see an overhaul is through the > strong willed determination of a few people off list, out of sight, to > allow it it to be fully fleshed out, with input coming from direct > consultation with and review by other related parties who have a vested > interested or significant experience in the area. > > I may be up for such an off list effort, but be warned I may want to run > roughshod over it and exert quite a lot of influence over the process and > outcome. :-) > > Graham > ___ > Web-SIG mailing list > Web-SIG@python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: > https://mail.python.org/mailman/options/web-sig/bchesneau%40gmail.com > ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] question about connection pool, task queue in WSGI
On Sun, Jul 15, 2012 at 5:14 PM, Simon Sapin wrote: > Le 14/07/2012 06:07, Graham Dumpleton a écrit : > >>2. Is the socket FD the same mechanism like nginx? If you upgrade >> nginx >>binary, restart nginx, the existing http connection won't break. >> >> I would be very surprised if you could upgrade nginx, perform a >> restart and preserve the HTTP listener socket. If you are talking >> about some other socket I don't know what you are talking about. >> >> As you can with Apache, you can likely enact a configuration file >> change and perform a restart or trigger rereading of the configuration >> and it would maintain the HTTP listener socket across the >> configuration restart, but an upgrade implies changing the binary and >> I know no way that you could easily persist a HTTP listener socket >> across to an invocation of a new web server instance using a new >> executable. In Apache you certainly cannot do it, and unless nginx has >> some magic where the existing nginx execs the new nginx version and >> somehow communicates through open socket connections to the new >> process, I very much doubt it would as it would be rather messy to do >> so. > > > I think that est refers to this: > http://wiki.nginx.org/CommandLine#Upgrading_To_a_New_Binary_On_The_Fly > > Apparently yes, there is specific code in nginx to start the new binary and > give it the existing socket. > > And I think that yes, Tarek’s new Circus is similar to the nginx magic > upgrade in that an open socket is passed around processes. Maybe nginx even > does this in normal operation with multiple worker processes, but I don’t > know. > > Regards, > -- > Simon Sapin Gunicorn does upgrade itself using the USR2 signal just like nginx and share the socket like using an fd between OS processes. However the case of a db is a little different since you handle a connection to a db and not listening on a port. You will need either a multiprocess queue passing messages to one process or launching a connection per processes. You can do that using the hook system of gunicorn. - benoît ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Starting Web Servers using socket FDs
On Tue, Jun 5, 2012 at 12:26 PM, Tarek Ziadé wrote: > On 6/5/12 11:46 AM, Roberto De Ioris wrote: >> >> ... >> >> Gunicorn can already bind (or better, accept) from file descriptors >> specifying an environment variable. > > I don't think you can start gunicorn using a file descriptor, or I failed to > do it. The best I was able to do was to create a small wsgi server > using Gunicorn as a lib. > > Gunicorn uses an environment variable when it respawns workers but it does > not offer it as a public option as far as I understand how it works > > export GUNICORN_FD= and then gunicorn will use this file descriptor when it starts. But it should be possible to pass it directly using a config option if it's needed. - benoît ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] A 'shutdown' function in WSGI
On Tue, Feb 21, 2012 at 10:35 AM, Sylvain Hellegouarch wrote: > > > On Tue, Feb 21, 2012 at 10:31 AM, Graham Dumpleton > wrote: >> >> On 21 February 2012 20:26, Simon Sapin wrote: >> > Le 21/02/2012 09:23, Tarek Ziadé a écrit : >> > >> >> Instead of having to provide two or three objects separately to a >> >> server, how about making the callbacks attributes of the application >> >> callable? >> >> >> >> >> >> can you show us an example ? >> > >> > >> > Proposal: >> > >> > Function-based: >> > >> > def startup(): >> > return open_resource(something) >> > >> > def shutdown(resource): >> > resource.close() >> > >> > def application(environ, start_response): >> > # ... >> > return response_body >> > >> > application.startup = startup >> > application.shutdown = shutdown >> > >> > Class-based: >> > >> > class App(object): >> > def startup(self): >> > return open_resource(something) >> > >> > def shutdown(self, resource): >> > resource.close() >> > >> > def __call__(self, environ, start_response): >> > # ... >> > return response_body >> > >> > application = App() >> > >> > The return value of startup() can be any python object and is opaque to >> > the >> > server. It is passed as-is to shutdown() >> > >> > startup() could take more parameters. Maybe the application (though can >> > we >> > already have it as self for class-based or in a closure for >> > function-based) >> >> You do realise you are just reinventing context managers? >> >> With this 'application' do requests. >> >> But then it was sort of suggested that was a bit too radical idea when >> I have mentioned viewing it that way before. :-( >> > > One might wonder if having access to process management should be part of > WSGI in the first place. > that's the thing. This is no more a gateway. Like I said in my previous post, maybe having another spec describing a web app package would do the trick? - benoît ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] A 'shutdown' function in WSGI
On Mon, Feb 20, 2012 at 5:03 PM, Tarek Ziadé wrote:
> Hello
>
> I need to be able to call a function when the web application shuts down
> (SIGTERM/SIGINT) -- the use case is to stop a background thread.
>
> I am currently using signals because it seems to be the most clean way to do
> this. atexit is much trickier since you don't know when it's going to get
> called and you might try to call objects that were garbage collected unless
> you hack something to keep references alive.
>
> But signals are also tricky beasts since you may compete with other code
> that are listening to them. For instance mod_wsgi don't like apps that have
> signal handlers.
>
> Anyways, the bottom line is that the cleanest way to do this -- as per Chris
> McDonough idea, would be to introduce in the WSGI protocol a "shutdown"
> function the servers would be obligated to call before exiting.
>
> I am not sure yet about its arguments, maybe a signum + frame or simply an
> exit code...
>
> But how do you like the idea ? That would solve for me the problem of
> having to deal differently here depending on if I am called with mod_wsgi or
> gunicorn or xxx
>
Hi Tarek,
In gunicorn we have the concept of hook which exactly does what you want
(on_exit hook). You can do this via the configuration file. And you can
do this for the other steps (reload, pre/post fork, pre/post request
...).
Anyway, I think there are 2 issues to solve with this shutdown thing
(and more) in a generic way:
1. how to make your application aware of the server capabilities (can it
or not handle shutdown)
2. how to pass such functions to the server.
There are also some others details like determining what is the context
needed by the application at this step.
Today the server is acting as a gateway so it just pass a request to an
application and return the response from the application. The
application by itself only lives in this context. Maybe, like some said,
if we want to give more info about that, we could imagine the application
object or function as a bus reacting on different contexts:
def application(environ, start_application):
pass
where the environ can be different depending on the context an action, we
could have:
http request{"wsgi.context": "http", ...}
shutdown{"wsgi.context"; "server", "action": "shutdown", ...}
smth like it. But how to know when the server can't pass the action
shutdown? Also what would be the environ at shutdown?
One other method, like the one described by Sylvain Hellegouarch, would
be adding some properties/attributes to the application object. So the
server would eventually check if they exists and eventually call them.
But again how to know when the server can't pass the action shutdown?
Though, I don't like to handle more than the application callable today,
I can see some problems when you have to handle server reload and such
things.
It seems important for me that the application knows about the server
capabilities if it want to handle such things like the `shutdown`. And
this step is solved by gunicorn & others by specifically handling a
configuration. But maybe we could just considers that a server always
handle the "reload", "shutdown", also pre/post "request" evenst and have
optionnals message passed to the application? (how do we handled process
context changes - if we re in a thread for ex - ?).
We could also imagine a web application package that provides hooks
handling depending on the server target... I quite like this idea.
Not sure what is the right way. What do you think?
- benoît
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe:
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] SERVER_PORT and Unix sockets
On Mon, Jan 2, 2012 at 3:59 PM, Jonas H. wrote: > Hello everyone! > > What is SERVER_PORT supposed to be set to if the WSGI server is only bound > to a Unix socket? > > Some major Web servers (Gunicorn, CherryPy) set it to the empty string. > Intuitively I'd rather not set it at all. > > What do you guys recommend? > > btw, www.wsgi.org != wsgi.org. That's very confusing. > > Jonas Mmm are you sure it's empty in gunicorn? For the server addres we are using socket.getsockname() for the default so at this step the port can be null, but then Host header (HTTP_HOST) which override the defaults. We are following the spec on that's matter. So I'm surprised that port can be empty. Can you provide an example I could reproduce? - benoit ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Declaring PEP 3333 accepted (was: PEP 444 != WSGI 2.0)
On Tue, Jan 4, 2011 at 12:13 AM, Jacob Kaplan-Moss wrote: > On Sun, Jan 2, 2011 at 9:21 AM, Guido van Rossum wrote: >> Although [PEP ] is still marked as draft, I personally think of it >> as accepted; [...] > > What does it take to get PEP formally marked as accepted? Is > there anything I can do to push that process forward? > > The lack of a WSGI answer on Py3 is the main thing that's keeping me, > personally, from feeling excited about the platform. Once that's done > I can feel comfortable coding to it -- and browbeating those who don't > support it. > > I understand that PEP 444/Web3/WSGI 2/whatever might be a better > answer, but it's clearly got some way to go. In the meantime, what's > next to get PEP officially endorsed and accepted? > > Jacob I we pep is considered accepted, I will start the port of gunicorn based on it too. It shouldn't be hard since we are using our own http parser. - benoit ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
[Web-SIG] Fwd: PEP 444 != WSGI 2.0
-- Forwarded message -- From: Benoit Chesneau Date: Sun, Jan 2, 2011 at 10:51 PM Subject: Re: [Web-SIG] PEP 444 != WSGI 2.0 To: Alice Bevan–McGregor On Sun, Jan 2, 2011 at 9:38 PM, Alice Bevan–McGregor wrote: > On 2011-01-02 11:14:00 -0800, Chris McDonough said: >>> >>> I'd suggest we just embrace it, adding minor tweaks as necessary, until >>> we reach some sort of technical impasse it doesn't address. > > Async is one area that does not cover, and that by not having a > standard which incorporates async means competing, incompatible solutions > have been created. > I don't see why async need a specific handling in the spec. What differences do you need ? Today users are perfectly abble to manage async using gevent, eventlet, ... behind WSGI 1.0 . - benoît ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Most WSGI servers close connections to early.
On Wed, Sep 22, 2010 at 5:34 PM, Robert Brewer wrote: > However, the caveat requires a caveat: servers must still be able to protect > themselves from malicious clients. In practice, that means allowing servers > to close the connection without reading the entire request body if a certain > number of bytes is exceeded. > I don't see how it could be the responsability of the server. Can you develop a little ? The server shouldn't interfere in the HTTP request imo. - benpît ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Most WSGI servers close connections to early.
On Wed, Sep 22, 2010 at 2:46 PM, Marcel Hellkamp wrote:
> I just discovered a problem that affects most WSGI server
> implementations and most current web-browsers (tested with wsgiref,
> paste, firefox, chrome, wget and curl):
>
> If the server closes the connection while the client is still uploading
> data via POST or PUT, the browser displays an error message ('Connection
> closed') and does not display the response sent by the server.
>
> The error occurs if an application chooses to not process a form
> submissions before returning to the WSGI server. This is quite rare in
> real world scenarios, but hard to debug because the server logs the
> request as successfully sent to the client.
>
> To reproduce the problem, run the following script, visit
> http://localhost:8080/ and upload a big file::
>
>
>
> from wsgiref.simple_server import make_server
>
> def application(environ, start_response):
> start_response('200 OK', [('Content-Type', 'text/html')])
> return ["""
>
> Upload bog file:
>
>
>
> """]
>
> server = make_server('localhost', 8080, application)
> server.serve_forever()
>
>
>
>
> I would like to add a warning to the WSGI/web3 specification to address
> this issue:
>
> "An application should read all available data from
> `environ['wsgi.input']` on POST or PUT requests, even if it does not
> process that data. Otherwise, the client might fail to complete the
> request and not display the response."
>
> --
> Mit freundlichen Grüßen
> Marcel Hellkamp
>
Your application and client should be aware of Expect: 100-Continue header :
http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html
- benoît
(resent, because web-sig doesn't set well the default reply-to)
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe:
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] PEP 444 (aka Web3)
On Thu, Sep 16, 2010 at 6:41 PM, Armin Ronacher wrote: >> 4. The web3 spec says, "In case a content length header is absent the >> stream must not return anything on read. It must never request more >> data than specified from the client." but later it says, "Web3 >> servers must handle any supported inbound "hop-by-hop" headers on >> their own, such as by decoding any inbound Transfer-Encoding, >> including chunked encoding if applicable.". I would be sad if web3 >> did not support streaming uploads via Transfer-Encoding. One way to >> implement that would be to make the origin server handle read() >> transparently by returning '' on EOF, regardless of whether a >> Content-Length or a Transfer-Encoding header was provided. > > I was toying with the idea to have a websocket extension for web3 which > would have solved my usecase for requests without a content-length header. > The problem with the content length of incoming data is quite complex and > that seemed to be the solution that was easiest for everybody involved. > uh ? Since with Transfer-Encoding: chunked we know when the stream end, I would be in favor of returning an EOF too at the end. Also most of servers know when a stream end even if there is no content-length. Maybe we could have a capability setting in environ that say if the server support streaming or not. And in all cases returning EOF at the end? - benoît ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] PEP 444 (aka Web3)
On Fri, Sep 17, 2010 at 10:36 AM, Georg Brandl wrote: > Am 16.09.2010 23:07, schrieb James Mills: >>> - the web3 name >>> >>> If there is any value in this PEP and we find something to decide on, >>> there is no reason this couldn't be WSGI 2. But until it's just >>> something a small part of the web-sig community worked on directly >>> a separate name is a good thing I think, because it does not reserve >>> the name "WSGI 2" for something that might actually become WSGI 2 >>> in case this PEP gets rejected. >> >> I personally still don't see any real benefit to changing the key names >> from "wsgi" to "web3" (or whatever). I would prefer it remain the >> same. If you're going to use Python3, you know you're using Python3 >> (you don't need "web3" key names to know that). (subjective) > > That statement shows another weakness of the "web3" name: this spec is not > in the least exclusive to Python 3. (Which would be a bit useless, having > two incompatible WSGI/web specs on two incompatible Python versions.) > > The goal would be to first migrate to WSGI2/web3, and *then* have an easy > transition going to Python 3. > > Georg > also WSGI acronym is defining better the purpose by itself than "web3" which mean nothing. - benoit ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] WSGI and start_response
On Thu, Apr 8, 2010 at 4:53 PM, P.J. Eby wrote: > At 04:08 PM 4/8/2010 +0200, Manlio Perillo wrote: >> >> Hi. >> >> Some time ago I objected the decision to remove start_response function >> from next version WSGI, using as rationale the fact that without >> start_callable, asynchronous extension are impossible to support. >> >> Now I have found that removing start_response will also make impossible >> to support coroutines (or, at least, some coroutines usage). >> >> Here is an example (this is the same example I posted few days ago): >> http://paste.pocoo.org/show/199202/ >> >> Forgetting about the write callable, the problem is that the application >> starts to yield data when tmpl.render_unicode function is called. >> >> Please note that this has *nothing* to do with asynchronus applications. >> The code should work with *all* WSGI implementations. >> >> >> In the pasted example, the Mako render_unicode function is "turned" into >> a generator, with a simple function that allows to flush the current >> buffer. >> >> >> Can someone else confirm that this code is impossible to support in WSGI >> 2.0? > > I don't understand why it's a problem. See my previous post here: > > http://mail.python.org/pipermail/web-sig/2009-September/003986.html > > for a sketch of a WSGI 1-to-2 converter. It takes a WSGI 1 application > callable as the input, and returns a WSGI 2 function. > where is WSGI 2 pep ? I would like to see it first rather than seeig different implementations. - benoit ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Migrating from mod_wsgi to FastCGI
On Mon, Mar 15, 2010 at 3:12 PM, Gustavo Narea wrote: > Hello, > > We're considering migrating from mod_wsgi to FastCGI (Apache) because we'll > need to use versions of Python compiled by ourselves. > > In addition to the research I've done and the pre-deployment tests we'll > carry out, I'd love to know if any of you has done the same migration. If > so, how was this experience? I suspect there's nothing to worry about > (that's what WSGI is for), but just in case... > > Thanks, > > - Gustavo. > > ___ > Web-SIG mailing list > Web-SIG@python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: > http://mail.python.org/mailman/options/web-sig/bchesneau%40gmail.com > > why not just use a gateway behind a proxy ? - benoit ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
[Web-SIG] Fwd: gunicorn 0.1 - new WSGI HTTP Server
-- Forwarded message -- From: Benoit Chesneau Date: Thu, Jan 21, 2010 at 4:12 PM Subject: Re: [Web-SIG] gunicorn 0.1 - new WSGI HTTP Server To: Tres Seaver On Wed, Jan 6, 2010 at 2:12 AM, Tres Seaver wrote: > Interesting: how are you detecting slow clients in production, given > that the WSGI server itself is only supposed to be used for fast ones? > I'm assuming that there must be some kind of heuristic-applying proxy > you run in front of unicorn/gunicorn. Or do you just not bother, and > let slow clients see failed responses? > > > Tres. Sorry for the delay to answer. Gunicorn intended to be used behind a caching upstream proxy like nginx. So it will be the responsability of nginx or such to manage slow connections which it does perfectly. In this sense it works like its alter ego unicorn on ruby. hope it helps. - benoît ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
[Web-SIG] gunicorn 0.1 - new WSGI HTTP Server
Hi, Quick mail to announce the gunicorn 'Green Unicorn' 0.1. it is a WSGI HTTP Server for UNIX, fast clients and nothing else. This is a port of Unicorn (http://unicorn.bogomips.org/) in Python. You can find it here : http://pypi.python.org/pypi/gunicorn/0.1 Current features are limited to the choice of number of workerts/cores you want to use and the ip/port. There are one command gunicorn_django that allows you to launch any django project. Any feedback is appreciated, - Benoit ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
[Web-SIG] process & ..
Hi all, As external I'm a bit surprised and also disappointed about actual process about choosing/discussing next features or changes of SGI spec. I thought and I think process should be more formal. Like I see it since some months, there are people trying to put their views and technical considerations but that's all. I don't see any vote about adding/editing current version of the spec. Some people try to summarize, but there is also not formal way. Also working on different repositories for one spec, with changes without votes make me think something is wrong. I don't see any step in discussion nor roadmap and a standard shouldn't be handled like this in my opinion. Maybe I'm wrong, maybe votes are hidden, there is something outside I ignore and I will be happy to be wrong on this. Anyway I really think in this case process should be more transparent. At least having a roadmap with formal steps like you have on other standards would help. The web is changing, and as a developer I would like to know what and when will be adapted WSGI to new needs. - benoit ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] String Types in WSGI [Graham's WSGI for py3]
On Sep 18, 2009, at 10:12 AM, René Dudfield wrote: Why is the raw url needed(very rarely)? Sometimes there are bugs. Access to the raw string lets you work around those bugs... if you need to. Dropping to a lower level is needed sometimes. Some APIs require you to send back an exact copy of the input url. Or sometimes you want to know what input url was used... not the cleaned up version of it. Sometimes clients calling the wsgi code will be buggy... and looking at the unquoted url is needed in those cases to work around buggy clients. And sometimes you need to support full uri spec. For example %2F is different from / . Actually if all url is decoded you don't know if the client request was %2F or /, you just get a /. Which is annoying. It causes some problem with some api ,I'm thinking to couchdb for example who accept db name with a %2F inside to allow creation of folder on user system. - benoit___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
