Re: Comet: problem with request.getParameter() in Comet POST requests
Hi, I am working on a scenario with browsers as clients. The client does requests with JavaScript code like this: req = new XMLHttpRequest(); req.onreadystatechange = handler; req.open("post", "/somecontext/somecometservlet/somerequest?param1=val1¶m2=val2" ); req.send(null); We intentionally use POST requests to avoid problems that occurred with some caches at customer sites and with web crawlers that monitor the GET requests of a browser and send the identical requests themselves to analyze the responses. Thus, only with POST we can achieve an "exactly once" submission of the request. However, by passing the request parameters in the URL (i.e. in the header and not in the body) I don't have any problems with reading the request parameters. You can imagine, that I would not be happy if the API would only offer me a request facade that throws an exception when the servlet tries to access the parameters of a POST request. Yes, I can imagine. It amazes me just what kind of hacks are sometimes necessary. :-) Anyway, after thinking on it some more, I think the best solution would be a fix so that getParameter() in a POST request does a blocking read, the same way the headers are read. If you do getParameter() in a POST that automatically interferes with getInputStream() anyway, and expect data to arrive pretty much immediately so there is not really a good reason for not blocking in this case. Regards, Sebastiaan - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Comet: problem with request.getParameter() in Comet POST requests
> -Original Message- > From: Rémy Maucherat [mailto:[EMAIL PROTECTED] > Sent: Wednesday, May 02, 2007 11:13 PM > On 5/2/07, Sebastiaan van Erk <[EMAIL PROTECTED]> wrote: > > It's a race condition and the problem occurs quite infrequently > > (especially with small request bodies). The larger the > request body of > > the POST request, the more likely it is that the problem > occurs. I was > > testing at a few thousand request per second, and even then > it rarely > > showed up. The fact that other people out there are using > getParameter() > > on Comet POST requests and expect it to work seems to me to > warrant 1) > > acknowledgement that it is a bug, 2) a warning in the > documentation, or > > 3) a request facade for the Comet API which throws an > exception when the > > getParameter() method is called on a POST request. > > Whatever ;) > > Rémy > I am working on a scenario with browsers as clients. The client does requests with JavaScript code like this: req = new XMLHttpRequest(); req.onreadystatechange = handler; req.open("post", "/somecontext/somecometservlet/somerequest?param1=val1¶m2=val2" ); req.send(null); We intentionally use POST requests to avoid problems that occurred with some caches at customer sites and with web crawlers that monitor the GET requests of a browser and send the identical requests themselves to analyze the responses. Thus, only with POST we can achieve an "exactly once" submission of the request. However, by passing the request parameters in the URL (i.e. in the header and not in the body) I don't have any problems with reading the request parameters. You can imagine, that I would not be happy if the API would only offer me a request facade that throws an exception when the servlet tries to access the parameters of a POST request. Matthias - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Comet: problem with request.getParameter() in Comet POST requests
> -Original Message- > From: Filip Hanik - Dev Lists [mailto:[EMAIL PROTECTED] > > Wouldn't every application which isn't as dumb as the chat example > > (which does not care about the content it reads but simply passes it > > back to it's clients) need to implement it's own mechanism to check > > whether there is enough input available to start parsing a chunk of > > data? > > > huh? This exact same problem exist with regular servlets, and has > existed for the last 8 years. > Filip > I have written some servlet code during the last years, but I never had to deal with the question if the servlet can start parsing - it simply does, and the underlying read operations on the stream block if necessary to wait until more data is available to proceed. (Or the servlet simply accesses request parameters of a POST request and expects that they are available.) The question whether I have to wait for another READ event before I have the complete request available (or a complete message from a series of messages transferred within the scope of one http request) only arises if I am told that I should not do a blocking read. So far I did not read a clear statement which says: With Comet is is not desirable to do blocking read, but the API allows to do it, as well in the BEGIN event handling as in the READ event handling. If this statement is true, I don't have any problem. The Servlet can read all it expects to be delivered by the client when the client starts a request (and don't care if it fits into one TCP packet or not, because the read will block until the remaining TCP packets have arrived.) Afterwards, there won't be anything more to be read until the client sends the next message. Thus, if the servlet gets a READ event, it can assume that the client has sent a new message and read it. (Again, the read will block if some TCP packets are missing which contain remaining parts of the message.) However, if the statement is not true, the Servlet has to care about what is going on on the transport (TCP) level, and it has to count bytes to know when a message is completely available. So can you tell me: Is the above statement true? Matthias - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Comet: problem with request.getParameter() in Comet POST requests
On 5/2/07, Sebastiaan van Erk <[EMAIL PROTECTED]> wrote: It's a race condition and the problem occurs quite infrequently (especially with small request bodies). The larger the request body of the POST request, the more likely it is that the problem occurs. I was testing at a few thousand request per second, and even then it rarely showed up. The fact that other people out there are using getParameter() on Comet POST requests and expect it to work seems to me to warrant 1) acknowledgement that it is a bug, 2) a warning in the documentation, or 3) a request facade for the Comet API which throws an exception when the getParameter() method is called on a POST request. Whatever ;) Rémy - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Comet: problem with request.getParameter() in Comet POST requests
Hi, I have no problem to get POST parameter with NIO Connector at my begin.event. With APR connector I also have POST parameter problems. Strange! A my first request I get the POST parameter. The POST message close the event and I got an exception after the end event. It's a race condition and the problem occurs quite infrequently (especially with small request bodies). The larger the request body of the POST request, the more likely it is that the problem occurs. I was testing at a few thousand request per second, and even then it rarely showed up. The fact that other people out there are using getParameter() on Comet POST requests and expect it to work seems to me to warrant 1) acknowledgement that it is a bug, 2) a warning in the documentation, or 3) a request facade for the Comet API which throws an exception when the getParameter() method is called on a POST request. The problem occurs when the post data is not yet received on the server when the getParameter() method is called. This is very easy to simulate: simply use telnet to your webserver port and copy paste the request headers (including the empty line), but WITHOUT the request body to your Comet servlet. Then you will see that you get a BEGIN event and the getParameter() call returns null. Then you can paste the rest of the request body, but it will be too late. If the request body is large than this problem is more likely to occur because the data is split over multiple TCP packets. Currently we have no IO Event, but Remy has wrote a proposal to dev list :-) I thing we need this feature. +1 Ah, I will browse the dev list archives; I have found a workaround currently, but I am interested in the developments for future projects. Regards, Sebastiaan - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Comet: problem with request.getParameter() in Comet POST requests
Hi Am 02.05.2007 um 17:26 schrieb Sebastiaan van Erk: Hi, How about not arguing about everything ? It is your fault when parameters are not processed. Tomcat will process parameters with comet. No, Comet will not process parameters in POST requests if you call getParameter() in the BEGIN event and the request body has not yet arrived. getParameter() will return null even if the parameter is set in the request body. How is this my fault? How do I know when to call getParameter()? How exactly am I supposed to process the POST parameters in a Comet request? I have no problem to get POST parameter with NIO Connector at my begin.event. With APR connector I also have POST parameter problems. Strange! A my first request I get the POST parameter. The POST message close the event and I got an exception after the end event. == This two messages are only exists at my patch tomcat trunk, but it show that POST read the 12 byte post body. 02.05.2007 20:24:00 org.apache.catalina.connector.Request parseParameters FEIN: before readPostBody 12 02.05.2007 20:24:00 org.apache.catalina.connector.Request parseParameters FEIN: post is [EMAIL PROTECTED] 12 == Asynch close of the event command=close Asynch close of the event Closing transport Asynch close of the event command=close Asynch close of the event Event received END Closing transport Leaving handle end 02.05.2007 20:24:04 org.apache.catalina.connector.CoyoteAdapter event SCHWERWIEGEND: The servlet did not read all available bytes during the processing of the read event Event received BEGIN Leaving handle begin Sending message Message0 Sending message Message1 Sending message Message2 Sending message Message3 Sending message Message4 Closing transport Event received END Closing transport Leaving handle end As I use the NIO connector I have no problems... Here my POST request: == POST /tcniotest/comet HTTP/1.1 Host: localhost:8080 User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; de; rv: 1.8.1.3) Gecko/20070309 Firefox/2.0.0.3 Accept: text/xml,application/xml,application/xhtml+xml,text/ html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 Accept-Language: de-de,de;q=0.8,en-us;q=0.5,en;q=0.3 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: close Content-Type: application/x-www-form-urlencoded X-Requested-With: XMLHttpRequest Referer: http://localhost:8080/tcniotest/ Content-Length: 12 Cookie: JSESSIONID=17451AD30A176E1B7E7292CD9E1A0987.tomcat6 Pragma: no-cache Cache-Control: no-cache action=close === Testet with current tomcat6 main trunk, java 6, mac osx 10.4.9 and firefox 2.0.0.3. , apr 1.2.8 tcnative 1.1.10 You need the IO event to free up more than just the memory. The file descriptor also needs to be freed; this was causing problems for me earlier. Then set a shorter timeout, allow an appropriate amount of file descriptors, or produce an IO event. It is possible that a notify method (or something like that) could be added on the event to make it a bit more straightforward. The first two are workarounds which start failing quite quickly when there is a high number of requests. As for the last option, how do "produce and IO event" on the input stream on the server side? It's the client that produces IO events on the input stream, and I have no control over the client. By the way, in RFC 2616 it says: HTTP/1.1 defines the "close" connection option for the sender to signal that the connection will be closed after completion of the response. It seems a good option that connection really close with the header "Connection: close" is set. For normal Htttp11Processor it is easy, but NIO and APR processor are more complex. Currently set a short timeout is the only option, but most programmes want to set Connection:close. So as far as I can tell it's not even required for the client to close the input stream after reading Content-Length bytes. It could very well wait for the server to close the request, which means that Comet receives no IO event and we end up having to wait for a TIMEOUT event. So please tell me how to "produce an IO event". Currently we have no IO Event, but Remy has wrote a proposal to dev list :-) I thing we need this feature. +1 The way to sync is according to what your application does. The chat mini example has a very simple sync structure. The sync depends on what the application is doing. The only official stance is that none of the structures that the Servlet API provides are synced. I don't see how this is enough information. Obviously I need to synchronize access to the output stream of the repsonse object. But internally, Tomcat uses this object too. For example, it closes the output stream in the event.close() method. Without me knowing when and how Tomcat uses this object, how am I supposed to know how to do the synchronization? For example, there is no explanation in the chat ex
Re: Comet: problem with request.getParameter() in Comet POST requests
Reich, Matthias wrote: -Original Message- From: Sebastiaan van Erk [mailto:[EMAIL PROTECTED] Sent: Friday, April 27, 2007 7:24 PM To: Tomcat Users List Subject: Re: Comet: problem with request.getParameter() in Comet POST requests GET parameters, ie parameters in the URL will work. However, using Comet you shouldn't rely on parameters in the body, the body if for you usage, and your usage alone. Seems to me that this is a pretty common use case though with AJAX server side push: request, wait, response. Comet is used to free the http thread while waiting, otherwise the server would be out of http threads in no time. The data posted is often in POST form (a scripted form post). You can still invoke getReader or getInputStream without actually reading from it, if that causes the parse to happen. Yes, that is my current workaround, I use READ events and the Content-Length header to parse the parameters myself. Wouldn't every application which isn't as dumb as the chat example (which does not care about the content it reads but simply passes it back to it's clients) need to implement it's own mechanism to check whether there is enough input available to start parsing a chunk of data? huh? This exact same problem exist with regular servlets, and has existed for the last 8 years. Filip According to my current understanding of Comet a READ event will be triggered after some tcp packet was received from the client. Thus, even if a client has sent a complete chunk of data (e.g. a SOAP request), the server side cannot be sure that it is completely available for parsing when a READ event is triggered. Thus, each application must implement it's own http like protocol to wrap each chunk of data transferred to the server: - The client must send the number of bytes of the following chunk before the actual chunk is sent. To be able to do this, it must care about content encoding (e.g. write character based data to a ByteArrayOutputStream first, determine it's length, send the number of bytes, and then the content of the byte array) - The servlet must read the length, allocate a byte buffer and start reading into the buffer. - Only if it has received a READ event that delivers the last expected byte of the chunk, the content of the byte buffer can be converted into a String (or another character based sequence) according to the character encoding specified by the client, and finally parsed. Is it really the intention of Comet that each application must care about these things? Is there no way to enable a Servlet to do a blocking read if it recognizes that a client has sent a chunk of data? It would not block for a long time in that situation (only until the missing tcp packets of the actual chunk have arrived). This would allow a much more convenient handling of input for most applications. Matthias - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Comet: problem with request.getParameter() in Comet POST requests
Hi, How about not arguing about everything ? It is your fault when parameters are not processed. Tomcat will process parameters with comet. No, Comet will not process parameters in POST requests if you call getParameter() in the BEGIN event and the request body has not yet arrived. getParameter() will return null even if the parameter is set in the request body. How is this my fault? How do I know when to call getParameter()? How exactly am I supposed to process the POST parameters in a Comet request? You need the IO event to free up more than just the memory. The file descriptor also needs to be freed; this was causing problems for me earlier. Then set a shorter timeout, allow an appropriate amount of file descriptors, or produce an IO event. It is possible that a notify method (or something like that) could be added on the event to make it a bit more straightforward. The first two are workarounds which start failing quite quickly when there is a high number of requests. As for the last option, how do "produce and IO event" on the input stream on the server side? It's the client that produces IO events on the input stream, and I have no control over the client. By the way, in RFC 2616 it says: HTTP/1.1 defines the "close" connection option for the sender to signal that the connection will be closed after completion of the response. So as far as I can tell it's not even required for the client to close the input stream after reading Content-Length bytes. It could very well wait for the server to close the request, which means that Comet receives no IO event and we end up having to wait for a TIMEOUT event. So please tell me how to "produce an IO event". The way to sync is according to what your application does. The chat mini example has a very simple sync structure. The sync depends on what the application is doing. The only official stance is that none of the structures that the Servlet API provides are synced. I don't see how this is enough information. Obviously I need to synchronize access to the output stream of the repsonse object. But internally, Tomcat uses this object too. For example, it closes the output stream in the event.close() method. Without me knowing when and how Tomcat uses this object, how am I supposed to know how to do the synchronization? For example, there is no explanation in the chat example why there is a synchronized (connections) block in the READ event. It looks like it's in there to syncrhonize access to the list, which could possibly be avoided with a CopyOnWriteArrayList. However, using a concurrent list and using synchronization on the output stream would break the chat program (because event.close() seems to internally access the output stream). Rémy Regards, Sebastiaan - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Comet: problem with request.getParameter() in Comet POST requests
On 5/2/07, Sebastiaan van Erk <[EMAIL PROTECTED]> wrote: Sorry about that, did not mean to say it is not possible to do a request, wait, response with Comet. All I'm trying to say is that Comet was not designed (at least, Filip stated this) for the async servlet model, and I tried to point out some of the issues when using Comet in It was not designed for that but it actually should work relatively easily. such a scenario, such as getParameter() not working, synchronization issues, etc.. There is a difference between "possible", and "easy/practical". What I meant was that the current API is not designed for both models, and it is not yet so practical for the async servlet model. How about not arguing about everything ? It is your fault when parameters are not processed. Tomcat will process parameters with comet. You need the IO event to free up more than just the memory. The file descriptor also needs to be freed; this was causing problems for me earlier. Then set a shorter timeout, allow an appropriate amount of file descriptors, or produce an IO event. It is possible that a notify method (or something like that) could be added on the event to make it a bit more straightforward. With a combination of "Content-Length" or chunked encoding plus a "Connection: close" header this should be possible, I agree. I would still like to know the "official" way to synchronize this is, though. The way to sync is according to what your application does. The chat mini example has a very simple sync structure. The sync depends on what the application is doing. The only official stance is that none of the structures that the Servlet API provides are synced. Rémy - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Comet: problem with request.getParameter() in Comet POST requests
Rémy Maucherat wrote: On 5/2/07, Sebastiaan van Erk <[EMAIL PROTECTED]> wrote: What you seem to want is more in line with the "asynchronous servlet" (request, wait, response), which Filip and Remy pointed out is not the quite the same as Tomcat's Comet. Ideally, both models would be possible through single unified API, but this is currently not yet the case. You're misrepresenting certain things. As you can see, it is already possible to do request, wait, response, Sorry about that, did not mean to say it is not possible to do a request, wait, response with Comet. All I'm trying to say is that Comet was not designed (at least, Filip stated this) for the async servlet model, and I tried to point out some of the issues when using Comet in such a scenario, such as getParameter() not working, synchronization issues, etc.. There is a difference between "possible", and "easy/practical". What I meant was that the current API is not designed for both models, and it is not yet so practical for the async servlet model. but you'll have to wait for an IO event to free up the memory used by the request processor. You need the IO event to free up more than just the memory. The file descriptor also needs to be freed; this was causing problems for me earlier. It would seem easy to me to cause the client to disconnect after the request, as keepalive is probably not extremely useful in this scenario, producing the event. With a combination of "Content-Length" or chunked encoding plus a "Connection: close" header this should be possible, I agree. I would still like to know the "official" way to synchronize this is, though. I've seen it happen that my application was still in the "write()" method of the reponse output stream when I already received the END event in my CometProcessor. At the moment I'm synchronizing event.close() and all access to the response output stream with the same lock, but I do not know for sure if this is the correct approach. I would very much like to know what the "official" approach is. Rémy Regards, Sebastiaan - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Comet: problem with request.getParameter() in Comet POST requests
On 5/2/07, Sebastiaan van Erk <[EMAIL PROTECTED]> wrote: What you seem to want is more in line with the "asynchronous servlet" (request, wait, response), which Filip and Remy pointed out is not the quite the same as Tomcat's Comet. Ideally, both models would be possible through single unified API, but this is currently not yet the case. You're misrepresenting certain things. As you can see, it is already possible to do request, wait, response, but you'll have to wait for an IO event to free up the memory used by the request processor. It would seem easy to me to cause the client to disconnect after the request, as keepalive is probably not extremely useful in this scenario, producing the event. Rémy - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Comet: problem with request.getParameter() in Comet POST requests
> > Wouldn't every application which isn't as dumb as the chat example > > (which does not care about the content it reads but simply passes it > > back to it's clients) need to implement it's own mechanism to check > > whether there is enough input available to start parsing a chunk of > > data? > > > Either that, or you can parse the information "on demand", as > it comes in. The problem is: I only know that SOME input is available, but I don't know if the parser will succeed without a blocking read. With other words: I cannot know whether the last closing bracket of my SOAP request is already available in the input unless I know the number of bytes of the complete chunk in advance. If I want to parse it 'on availability' (I think 'on demand' is not the right term here) I would need a parser that suspends on reaching the end of available data (returning something like a 'not finished' indication) and can be resumed with the next read event. Otherwise I would have to try a new parse attempt (starting with byte 0 again) with each read event, until I finally succeed. Furthermore, I usually parse a sequence of characters, not bytes, and I think it does not make sense to start decoding the bytes in the input if there are still bytes of the message missing. Thus, my conclusion is that 'parsing on availability' is not a solution except for some rare situations. > What you seem to want is more in line with the "asynchronous servlet" Well, for the scenario I am working on, you are right. But the problem of knowing whether a complete chunk of input is available for processing without a blocking read is also present if a Servlet reads one SOAP request after the other within the scope of the same http request. > As for blocking reads in your servlet, I don't think there is > anything > which prevents you from doing multiple blocking reads from > the request > input stream in a READ event. The description of the READ event is as > follows: > > EventType.READ: This indicates that input data is available, and that > one read can be made without blocking. > > It does not say you are not allowed to do additional reads (with the > risk of blocking). Note that I have not tried this yet, and > it might not > work. It would be useful to have a thorough description of > the API which > documents these things (as well as other issues such as > synchronization, > lifetime/scope of the event object, etc), with a clear seperation > between API (things one can depend on), and implementation details > (things which happen to work some way due to the current > implementation > but are subject to change without notice). I totally agree! Matthias - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Comet: problem with request.getParameter() in Comet POST requests
Hi, Wouldn't every application which isn't as dumb as the chat example (which does not care about the content it reads but simply passes it back to it's clients) need to implement it's own mechanism to check whether there is enough input available to start parsing a chunk of data? Either that, or you can parse the information "on demand", as it comes in. Is it really the intention of Comet that each application must care about these things? The input part of Comet allows you to process input as it comes in. This necessarily makes matters more complex than a blocking read. Is there no way to enable a Servlet to do a blocking read if it recognizes that a client has sent a chunk of data? It would not block for a long time in that situation (only until the missing tcp packets of the actual chunk have arrived). This would allow a much more convenient handling of input for most applications. Matthias What you seem to want is more in line with the "asynchronous servlet" (request, wait, response), which Filip and Remy pointed out is not the quite the same as Tomcat's Comet. Ideally, both models would be possible through single unified API, but this is currently not yet the case. The reason I was wondering about the getParameters() method is because it is a part of the Servlet API (specifically, the request object), and it SEEMS to work in Comet, but there is no guarantee that it will. Either this could be considered to be a bug (a fix being that getParameter() blocks until all parameter data is in), or it should be documented somewhere that getParameter() should not be used in the CometProcessor. As for blocking reads in your servlet, I don't think there is anything which prevents you from doing multiple blocking reads from the request input stream in a READ event. The description of the READ event is as follows: EventType.READ: This indicates that input data is available, and that one read can be made without blocking. It does not say you are not allowed to do additional reads (with the risk of blocking). Note that I have not tried this yet, and it might not work. It would be useful to have a thorough description of the API which documents these things (as well as other issues such as synchronization, lifetime/scope of the event object, etc), with a clear seperation between API (things one can depend on), and implementation details (things which happen to work some way due to the current implementation but are subject to change without notice). Regards, Sebastiaan - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Comet: problem with request.getParameter() in Comet POST requests
> -Original Message- > From: Sebastiaan van Erk [mailto:[EMAIL PROTECTED] > Sent: Friday, April 27, 2007 7:24 PM > To: Tomcat Users List > Subject: Re: Comet: problem with request.getParameter() in > Comet POST requests > > > > GET parameters, ie parameters in the URL will work. > > However, using Comet you shouldn't rely on parameters in > the body, the > > body if for you usage, and your usage alone. > Seems to me that this is a pretty common use case though with AJAX > server side push: request, wait, response. > Comet is used to free the http thread while waiting, otherwise the > server would be out of http threads in no time. > The data posted is often in POST form (a scripted form post). > > You can still invoke getReader or getInputStream without actually > > reading from it, if that causes the parse to happen. > Yes, that is my current workaround, I use READ events and the > Content-Length header to parse the parameters myself. Wouldn't every application which isn't as dumb as the chat example (which does not care about the content it reads but simply passes it back to it's clients) need to implement it's own mechanism to check whether there is enough input available to start parsing a chunk of data? According to my current understanding of Comet a READ event will be triggered after some tcp packet was received from the client. Thus, even if a client has sent a complete chunk of data (e.g. a SOAP request), the server side cannot be sure that it is completely available for parsing when a READ event is triggered. Thus, each application must implement it's own http like protocol to wrap each chunk of data transferred to the server: - The client must send the number of bytes of the following chunk before the actual chunk is sent. To be able to do this, it must care about content encoding (e.g. write character based data to a ByteArrayOutputStream first, determine it's length, send the number of bytes, and then the content of the byte array) - The servlet must read the length, allocate a byte buffer and start reading into the buffer. - Only if it has received a READ event that delivers the last expected byte of the chunk, the content of the byte buffer can be converted into a String (or another character based sequence) according to the character encoding specified by the client, and finally parsed. Is it really the intention of Comet that each application must care about these things? Is there no way to enable a Servlet to do a blocking read if it recognizes that a client has sent a chunk of data? It would not block for a long time in that situation (only until the missing tcp packets of the actual chunk have arrived). This would allow a much more convenient handling of input for most applications. Matthias - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Comet: problem with request.getParameter() in Comet POST requests
Rémy Maucherat wrote: On 4/27/07, Sebastiaan van Erk <[EMAIL PROTECTED]> wrote: > GET parameters, ie parameters in the URL will work. > However, using Comet you shouldn't rely on parameters in the body, the > body if for you usage, and your usage alone. Seems to me that this is a pretty common use case though with AJAX server side push: request, wait, response. Comet is used to free the http thread while waiting, otherwise the server would be out of http threads in no time. The data posted is often in POST form (a scripted form post). This is not very useful: threads are fairly cheap, and having all the plumbing to keep all the request data in memory and ready to use asynchronously is not exactly free (it makes the complexity an order of magnitude higher, also). If all you want to do is wait a while and still support 1000s of connections, then you should consider using threads and buying an extra 1GB of ram on the server, it might be a better deal. for now that is definitely a viable work around. however "threads being cheap" is a relative term, considering most threads run 128-256k/thread in the native memory space (outside the -Xmx area) cheap becomes resource expensive really quick. same thing with the context switching. Filip Rémy - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Comet: problem with request.getParameter() in Comet POST requests
On 4/27/07, Sebastiaan van Erk <[EMAIL PROTECTED]> wrote: > GET parameters, ie parameters in the URL will work. > However, using Comet you shouldn't rely on parameters in the body, the > body if for you usage, and your usage alone. Seems to me that this is a pretty common use case though with AJAX server side push: request, wait, response. Comet is used to free the http thread while waiting, otherwise the server would be out of http threads in no time. The data posted is often in POST form (a scripted form post). This is not very useful: threads are fairly cheap, and having all the plumbing to keep all the request data in memory and ready to use asynchronously is not exactly free (it makes the complexity an order of magnitude higher, also). If all you want to do is wait a while and still support 1000s of connections, then you should consider using threads and buying an extra 1GB of ram on the server, it might be a better deal. Rémy - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Comet: problem with request.getParameter() in Comet POST requests
GET parameters, ie parameters in the URL will work. However, using Comet you shouldn't rely on parameters in the body, the body if for you usage, and your usage alone. Seems to me that this is a pretty common use case though with AJAX server side push: request, wait, response. Comet is used to free the http thread while waiting, otherwise the server would be out of http threads in no time. The data posted is often in POST form (a scripted form post). You can still invoke getReader or getInputStream without actually reading from it, if that causes the parse to happen. Yes, that is my current workaround, I use READ events and the Content-Length header to parse the parameters myself. I still need to rethink the "no read event" when the body comes in with the request package. btw, Remy is working on improving the available() method to actually return data should there be some, this will take care of the sentence above and also work around the blocking reads Filip Currently I'm using the workaround suggested, I just do a read in the BEGIN event (I need to do that for the parameter parsing as well). I'd be really happy if the available() method returned >0 if there was data, that would already mean that the above workaround is failsafe (i.e., no blocking). As for the final solution, the current documentation on the website is a bit ambiguous: it says in the READ event description, "It is not allowed to attempt reading data from the request object outside of the execution of this method." However, "this method" is the event method, so it could be interpreted as meaning that it's OK to read in the BEGIN event. Personally I think the API would be cleanest if it was not confused with the current implementation, that is, that the event method is called with a READ directly after BEGIN if available() > 0. READ means "data available" (according to the same docs), and even though no extra I/O event occurred on the socket, for the user of the API that condition is met. Thanks for all the replies and fixes so far! Sebastiaan - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Comet: problem with request.getParameter() in Comet POST requests
Sebastiaan van Erk wrote: Rémy Maucherat wrote: On 4/26/07, Sebastiaan van Erk <[EMAIL PROTECTED]> wrote: Hi, If the body of the POST request is not present at the time of the getParameter() call, it returns null even if the parameter value is set. Things don't work like this, obviously. Parameter parsing only occurs if nothing used getReader or getInputStream. There's nothing to document here: it will simply not parse. There IS something to document. Please think yourself as well! Nothing did use the getReader() or getInputStream() method. I'm talking about using getParameter() *FIRST* thing in the BEGIN event. GET parameters, ie parameters in the URL will work. However, using Comet you shouldn't rely on parameters in the body, the body if for you usage, and your usage alone. You can still invoke getReader or getInputStream without actually reading from it, if that causes the parse to happen. I still need to rethink the "no read event" when the body comes in with the request package. btw, Remy is working on improving the available() method to actually return data should there be some, this will take care of the sentence above and also work around the blocking reads Filip Regards, Sebastiaan I think you should think over your emails a little bit more before posting. And I think you should think over my emails a little bit more before posting. Also it would be nice if you answered some of the simple yes/no questions and not be so hostile. Regards, Sebastiaan Rémy - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Comet: problem with request.getParameter() in Comet POST requests
Rémy Maucherat wrote: On 4/26/07, Sebastiaan van Erk <[EMAIL PROTECTED]> wrote: Hi, If the body of the POST request is not present at the time of the getParameter() call, it returns null even if the parameter value is set. Things don't work like this, obviously. Parameter parsing only occurs if nothing used getReader or getInputStream. There's nothing to document here: it will simply not parse. There IS something to document. Please think yourself as well! Nothing did use the getReader() or getInputStream() method. I'm talking about using getParameter() *FIRST* thing in the BEGIN event. Regards, Sebastiaan I think you should think over your emails a little bit more before posting. And I think you should think over my emails a little bit more before posting. Also it would be nice if you answered some of the simple yes/no questions and not be so hostile. Regards, Sebastiaan Rémy - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Comet: problem with request.getParameter() in Comet POST requests
On 4/26/07, Sebastiaan van Erk <[EMAIL PROTECTED]> wrote: Hi, If the body of the POST request is not present at the time of the getParameter() call, it returns null even if the parameter value is set. Things don't work like this, obviously. Parameter parsing only occurs if nothing used getReader or getInputStream. There's nothing to document here: it will simply not parse. I think you should think over your emails a little bit more before posting. Rémy - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]