Re: [Web-SIG] [RFC] x-wsgiorg.suspend extension
Graham Dumpleton ha scritto: > On 12 April 2010 06:07, Manlio Perillo wrote: >> I'm not sure about the correct procedure to follow, I hope it is not a >> problem. >> >> I here propose the x-wsgiorg.suspend to be accepted as official WSGI >> extension, using the wsgiorg namespace. >> First of all thanks for the feedback. > [...] > In the code of demo_fdevent.py it has: > > while True: > while True: > ret, num_handles = m.perform() > if ret != pycurl.E_CALL_MULTI_PERFORM: > break > if not num_handles: > break > > read, write, exc = m.fdset() > resume = environ['x-wsgiorg.suspend'](1000) > if read: > readable(read[0], resume) > yield '' > else: > writeable(write[0], resume) > yield '' > > The registration of file descriptors doesn't occur until after the > first suspend() call. > > If the underlying reactor that the WSGI server is presumably also > using doesn't know about the file descriptors at that point, then how > does it now to return from the suspend(). > I'm not sure to understand your concern, but the execution is not suspended when you call x-wsgiorg.suspend, but only when you yield a empty string. In the example, registration of file descriptor occur before application is suspended. > You are also calling perform() before that point. When calling that, > it is presumed you have already done a select/poll to know data is > available, but you haven't done that on first pass through the loop. > If you call that and data isn't ready, can't it block still. > I have to admit that I just copied the example from fdevent specification. However the code seems correct, to me. > This example also illustrates well why I am so against an asynchronous > WSGI server extension. > > The reason is that your specific application has to be with this > extension bound to the specific event loop mechanism used by the > underlying WSGI server. > > I can't for example take this application and host it on a different > WSGI server which implements the same WSGI extension but uses a > different event loop. > Instead I think that being "agnostic" about how it is used, in one of the most important feature of x-wsgiorg.suspend extension. After all, if you think about it, how to interface with a database in a WSGI application is not specified by WSGI. This is done by a separate standard, dbapi2. For applications that need a template engine, we don't even have a standard inteface. The lack of a standard event API is not a problem that should be discussed in WSGI. It is a problem with the Python community; in fact I would like to define a standard event API *and* a standard efficient network API (the reason is expressed at the end of the README file in txwsgi). > If one can't do that and it is tied to the event loop and > infrastructure of the underlying WSGI server, what is the point of > defining and implementing the WSGI extension as it doesn't aid > portability at all, so what service is it actually providing? > The service it provides is: "allow a WSGI application to suspend its execution and resume it later". > In that respect, the extension: > > http://www.wsgi.org/wsgi/Specifications/fdevent/ > > provided more as at least it tried to abstract out a generic interface > for registering interest in file descriptor activity and so perhaps > allow the application not to be dependent on the specific event loop > used by the underlying WSGI server. > However exposing this event interface is really something that has little to do with WSGI. Moreover, the fdevent example is rather inefficient. Suspensions should be minimized, and this is not possible with x-wsgiorg.fdevent but it is possible with x-wsgiorg.suspend. >>From the open issues of that other specification however, you can see > that there can be problems. It only allowed an application to be > interested in a single file descriptor where some packages may need to > express interest in more than one. > > Quite often an application is never going to be that simple anyway. > Some event systems allow a lot more than just watching of file > descriptors and timeouts however. You cant come up with a generic > interface for all these as they will not be able to be implemented by > a different event system which isn't so feature rich or which has a > different style of interface. Thus applications are restricted to the > lowest common denominator and likely that is not going to be enough > for most and so have no choice but to bind it to interfaces of > specific event loop. This is the reason why x-wsgiorg.resume is a better API than the one proposed by x-wsgiorg.fdevent, IMHO. > If that is going to be the case anyway, you may > as well forget about WSGI and write to that event systems specific web > server interface. > > So, given that one of the strengths of WSGI is that it is an interface > wh
Re: [Web-SIG] [RFC] x-wsgiorg.suspend extension
On 12 April 2010 21:25, Manlio Perillo wrote: > Graham Dumpleton ha scritto: >> On 12 April 2010 06:07, Manlio Perillo wrote: >>> I'm not sure about the correct procedure to follow, I hope it is not a >>> problem. >>> >>> I here propose the x-wsgiorg.suspend to be accepted as official WSGI >>> extension, using the wsgiorg namespace. >>> > > First of all thanks for the feedback. > >> [...] >> In the code of demo_fdevent.py it has: >> >> while True: >> while True: >> ret, num_handles = m.perform() >> if ret != pycurl.E_CALL_MULTI_PERFORM: >> break >> if not num_handles: >> break >> >> read, write, exc = m.fdset() >> resume = environ['x-wsgiorg.suspend'](1000) >> if read: >> readable(read[0], resume) >> yield '' >> else: >> writeable(write[0], resume) >> yield '' >> >> The registration of file descriptors doesn't occur until after the >> first suspend() call. >> >> If the underlying reactor that the WSGI server is presumably also >> using doesn't know about the file descriptors at that point, then how >> does it now to return from the suspend(). >> > > I'm not sure to understand your concern, but the execution is not > suspended when you call x-wsgiorg.suspend, but only when you yield a > empty string. Okay, missed that. > In the example, registration of file descriptor occur before application > is suspended. > >> You are also calling perform() before that point. When calling that, >> it is presumed you have already done a select/poll to know data is >> available, but you haven't done that on first pass through the loop. >> If you call that and data isn't ready, can't it block still. >> > > I have to admit that I just copied the example from fdevent specification. > However the code seems correct, to me. > >> This example also illustrates well why I am so against an asynchronous >> WSGI server extension. >> >> The reason is that your specific application has to be with this >> extension bound to the specific event loop mechanism used by the >> underlying WSGI server. >> >> I can't for example take this application and host it on a different >> WSGI server which implements the same WSGI extension but uses a >> different event loop. >> > > Instead I think that being "agnostic" about how it is used, in one of > the most important feature of x-wsgiorg.suspend extension. > > After all, if you think about it, how to interface with a database in a > WSGI application is not specified by WSGI. > This is done by a separate standard, dbapi2. > > For applications that need a template engine, we don't even have a > standard inteface. > > The lack of a standard event API is not a problem that should be > discussed in WSGI. > It is a problem with the Python community; in fact I would like to > define a standard event API *and* a standard efficient network API (the > reason is expressed at the end of the README file in txwsgi). > >> If one can't do that and it is tied to the event loop and >> infrastructure of the underlying WSGI server, what is the point of >> defining and implementing the WSGI extension as it doesn't aid >> portability at all, so what service is it actually providing? >> > > The service it provides is: "allow a WSGI application to suspend its > execution and resume it later". > >> In that respect, the extension: >> >> http://www.wsgi.org/wsgi/Specifications/fdevent/ >> >> provided more as at least it tried to abstract out a generic interface >> for registering interest in file descriptor activity and so perhaps >> allow the application not to be dependent on the specific event loop >> used by the underlying WSGI server. >> > > However exposing this event interface is really something that has > little to do with WSGI. > > Moreover, the fdevent example is rather inefficient. > Suspensions should be minimized, and this is not possible with > x-wsgiorg.fdevent but it is possible with x-wsgiorg.suspend. > >>>From the open issues of that other specification however, you can see >> that there can be problems. It only allowed an application to be >> interested in a single file descriptor where some packages may need to >> express interest in more than one. >> >> Quite often an application is never going to be that simple anyway. >> Some event systems allow a lot more than just watching of file >> descriptors and timeouts however. You cant come up with a generic >> interface for all these as they will not be able to be implemented by >> a different event system which isn't so feature rich or which has a >> different style of interface. Thus applications are restricted to the >> lowest common denominator and likely that is not going to be enough >> for most and so have no choice but to bind it to interfaces of >> specific event loop. > > This is the reason why x-wsgiorg.resume is a better API than the one > proposed by x-wsgiorg.fdevent, IMHO. > >> If that is g
Re: [Web-SIG] [RFC] x-wsgiorg.suspend extension
Graham Dumpleton ha scritto: > [...] >> >> Claiming that x-wsgiorg.suspend does not help writing portable WSGI >> application is something similar (well, I'm a bit exaggerating here) of >> saying that WSGI does not allow to write portable web applications, >> because real world WSGI applications needs a database, a database >> engine, and so on. > > It is not the same. I can take code using a specific database instance > and still run that WSGI application, using the same database, on a > different WSGI hosting mechanism without really changing anything > about how I interact with the WSGI server and its request handling. > The concern here is the WSGI interface and interacting with the web > server, not other non related third party packages. > This is true. However you can say the same for x-wsgorg.suspend extension. As an example, you can have an application that use a standard event API, and you can run it on several asynchronous WSGI implementations. The difference is that here we speak about event API, and not specific event implementation. Note however that we can also speak about specific implementations. As an example, I can implement Twisted reactor API in Nginx, so that WSGI applications using Twisted API can be executed on both Twisted and Nginx. I could do the same with libevent API. It's only a technical problem. > You are articificially adding something to the WSGI interface as an > extension which is pointless. Since you are bound to the specific > event loop of the underlying WSGI server or event framework being used You are not bound to a specific event framework, when using x-wsgiorg.suspend! > you may just as well call a function directly on the WSGI server. > Adding that function under a key in the WSGI environment and accessing > it that way does not in itself provide any value and doesn't somehow > make the code easily portable to a different WSGI hosting mechanism > using a different event loop as you still have to change lots of other > code in your application. > This is absolutely not true! > In some respects this is similar to the issues between using a WSGI > wrapper which injects stuff in WSGI environment versus that > functionality being in a separate library. Read: > > http://dirtsimple.org/2007/02/wsgi-middleware-considered-harmful.html > This is simply wrong. x-wsgiorg.suspend **can not** be implemented as simply library code; it **must** be accessed from environ dictionary. The reason is simple: 1) First of all, in order to suspend application, you **must** return control to the server, and this can only be done by yielding some value in the application generator. 2) In order for the implementation to know if application requested suspension, it must keep a flag in its *internal* state. The x-wsgiorg.suspend function simply sets this flag. Manlio ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] [RFC] x-wsgiorg.suspend extension
At 01:25 PM 4/12/2010 +0200, Manlio Perillo wrote: The purpose of the extension if to just have a standard interface that WSGI applications can use to take advantage of the possibility, offered by asynchronous server, to suspend execution and resume it later. WSGI has this ability now - it's yielding an empty string. Yielding an empty string is a hint to the server that the application is not ready to send any output, and the server is free to schedule other applications next. And WSGI does not require the application to be rescheduled any time soon. In other words, if saying "don't call me for a while" is the purpose of the extension, it is not needed. As Graham says, the thing that would actually be needed is a way to tell the server when to poll the app again. ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] [RFC] x-wsgiorg.suspend extension
P.J. Eby ha scritto: > At 01:25 PM 4/12/2010 +0200, Manlio Perillo wrote: >> The purpose of the extension if to just have a standard interface that >> WSGI applications can use to take advantage of the possibility, offered >> by asynchronous server, to suspend execution and resume it later. > > WSGI has this ability now - it's yielding an empty string. Yielding an > empty string is a hint to the server that the application is not ready > to send any output, and the server is free to schedule other > applications next. And WSGI does not require the application to be > rescheduled any time soon. > > In other words, if saying "don't call me for a while" is the purpose of > the extension, it is not needed. As Graham says, the thing that would > actually be needed is a way to tell the server when to poll the app again. > Just yielding an empty string does not give the server some important informations. As an example, with x-wsgi.suspend application can specify a timeout, that tells the server that the application must be resumed before timeout milliseconds have elapsed. And x-wsgi.suspend returns a callable that, when called, tell the server to poll the app again. Regards Manlio ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] [RFC] x-wsgiorg.suspend extension
On 13 April 2010 00:39, Manlio Perillo wrote: > P.J. Eby ha scritto: >> At 01:25 PM 4/12/2010 +0200, Manlio Perillo wrote: >>> The purpose of the extension if to just have a standard interface that >>> WSGI applications can use to take advantage of the possibility, offered >>> by asynchronous server, to suspend execution and resume it later. >> >> WSGI has this ability now - it's yielding an empty string. Yielding an >> empty string is a hint to the server that the application is not ready >> to send any output, and the server is free to schedule other >> applications next. And WSGI does not require the application to be >> rescheduled any time soon. >> >> In other words, if saying "don't call me for a while" is the purpose of >> the extension, it is not needed. As Graham says, the thing that would >> actually be needed is a way to tell the server when to poll the app again. >> > > Just yielding an empty string does not give the server some important > informations. > > As an example, with x-wsgi.suspend application can specify a timeout, > that tells the server that the application must be resumed before > timeout milliseconds have elapsed. > > And x-wsgi.suspend returns a callable that, when called, tell the server > to poll the app again. There are other ways of doing that, the callable doesn't need to be in the WSGI environment. This is because since it is single threaded, the WSGI server need only record in a global variable for that WSGI application some state about the current request. The separate function to note the suspension can then lookup that and does what it needs to. In other words, you don't need the WSGI environment to maintain that relationship. Having the timeout as argument is also questionable anyway. All you really need to do is to tell the WSGI server that I don't want to be called until I tell it otherwise. The WSGI application could itself handle the timeout in other ways. Overall one could do all of this without having to do anything in the WSGI environment. As PJE points out, it can be done by relying only on the ability to yield an empty string. Everything else can be in the application realm with the application normally being bound to a specific WSGI server/event loop implementation, thus no portability. The problem of a middleware not passing through an empty string doesn't even need to be an issue in as much as the application could track when it requested to be suspended and if called into again before the required criteria had been met, it could detect a middleware that wasn't playing by the rules and at least raise an error rather than potentially go into blocking state and tight loop. One could theoretically abstract out an interface for a generic event system, but what you don't want is a general purpose one. You want one which is specifically associated with the concept of a WSGI server. That way the API for it can expose methods which specifically relate to stuff like suspension of calling into the WSGI application for data until specific events occur. That abstract interface could then be implemented as concrete implementations for specific event based WSGI servers. Because a handle to that instance would be needed by the application, including outside context of a request, then a requirement of the interface may be that this handle to the WSGI server event interface be passed as argument to the WSGI application when it is created. So, you could come up with a standard for asynchronous WSGI, but the WSGI specification itself doesn't need to change nor additional keys put in the WSGI environment. Instead, any standardised interfaces exist outside of that and relates more to the interaction between application and underlying WSGI server directly, independent of a specific request in the large part. Graham ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
