Just to make sure that I wasn't crazy, I did some more tests,
including using your cometgui.jar. My results still show both read
errors and END events. Is it possible that it's a platform issue?
I'm running Windows XP Pro SP2. As I mentioned before I'm using the
tomcat 6.0.x trunk as of 21-01-2008. Also, I re-started tomcat in
between all tests to make sure there weren't stray bytes hanging
around somewhere. My server code is the same as shown previously in
this thread.
Peter
Here are my results:
1) Running LastChunkTest (code below) with last chunk of 0crlfcrlf,
server shows:
event: BEGIN, subtype: null
event: READ, subtype: null
Read 10 bytes: comet test for session: 627378C9DEE2817A93EBAC190DE47599
read error
2) Running LastChunkTest with last chunk of 0crlfcrlf and a 50 ms
sleep before sending last chunk, server shows:
event: BEGIN, subtype: null
event: READ, subtype: null
Read 10 bytes: comet test for session: A00EFFC9A9FE8CBF1833D204EF00667C
event: END, subtype: null
3) Running LastChunkTest with a last chunk of 0crlf, server shows:
event: BEGIN, subtype: null
event: READ, subtype: null
Read 10 bytes: comet test for session: DD41C198E3D5CD6E7D73FC0D7B37E86F
4) Running cometgui.jar with 0crlfcrlf:
client sends (without the dashes):
---
GET /CometTest HTTP/1.1
User-Agent: Filips Awesome Client/1.0
Host: 127.0.0.1:8080
Transfer-Encoding: chunked
10
test data test 1
0
---
server shows:
event: BEGIN, subtype: null
event: READ, subtype: null
Read 16 bytes: test data test 1 for session: 4023C27063C5674C1F37D315C3AF0AFC
read error
5) Running cometgui.jar with 0crlf:
client sends (without the dashes):
---
GET /CometTest HTTP/1.1
User-Agent: Filips Awesome Client/1.0
Host: 127.0.0.1:8080
Transfer-Encoding: chunked
10
test data test 1
0
---
server shows:
event: BEGIN, subtype: null
event: READ, subtype: null
Read 16 bytes: test data test 1 for session: BC5EA8DA373D1E6726020F44F71165C7
side note: Not sure if this is important, but when doing the following
steps in cometgui for the client data shown above, I get the exception
below:
connect
submit
disconnect
Jan 22, 2008 10:42:49 AM org.apache.catalina.core.StandardWrapperValve event
SEVERE: Servlet.service() for servlet CometTest threw exception
java.io.EOFException: Unexpected EOF read on the socket
at org.apache.coyote.http11.InternalNioInputBuffer.readSocket(InternalNi
oInputBuffer.java:589)
at org.apache.coyote.http11.InternalNioInputBuffer.fill(InternalNioInput
Buffer.java:868)
at org.apache.coyote.http11.InternalNioInputBuffer$SocketInputBuffer.doR
ead(InternalNioInputBuffer.java:892)
at org.apache.coyote.http11.filters.ChunkedInputFilter.readBytes(Chunked
InputFilter.java:243)
at org.apache.coyote.http11.filters.ChunkedInputFilter.parseCRLF(Chunked
InputFilter.java:326)
at org.apache.coyote.http11.filters.ChunkedInputFilter.parseEndChunk(Chu
nkedInputFilter.java:356)
at org.apache.coyote.http11.filters.ChunkedInputFilter.doRead(ChunkedInp
utFilter.java:136)
at org.apache.coyote.http11.InternalNioInputBuffer.doRead(InternalNioInp
utBuffer.java:830)
at org.apache.coyote.Request.doRead(Request.java:428)
at org.apache.catalina.connector.InputBuffer.realReadBytes(InputBuffer.j
ava:298)
at org.apache.tomcat.util.buf.ByteChunk.substract(ByteChunk.java:405)
at org.apache.catalina.connector.InputBuffer.read(InputBuffer.java:313)
at org.apache.catalina.connector.CoyoteInputStream.read(CoyoteInputStrea
m.java:162)
at com.seekspeak.server.debug.CometTestServlet.event(CometTestServlet.ja
va:31)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilterEvent
(ApplicationFilterChain.java:470)
at org.apache.catalina.core.ApplicationFilterChain.doFilterEvent(Applica
tionFilterChain.java:363)
at org.apache.catalina.core.StandardWrapperValve.event(StandardWrapperVa
lve.java:422)
at org.apache.catalina.core.StandardContextValve.event(StandardContextVa
lve.java:252)
at org.apache.catalina.core.StandardHostValve.event(StandardHostValve.ja
va:179)
at org.apache.catalina.valves.ValveBase.event(ValveBase.java:200)
at org.apache.catalina.valves.ValveBase.event(ValveBase.java:200)
at org.apache.catalina.core.StandardEngineValve.event(StandardEngineValv
e.java:128)
at org.apache.catalina.connector.CoyoteAdapter.event(CoyoteAdapter.java:
198)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.jav
a:292)
at org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcesso
r.java:876)
at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.pr
ocess(Http11NioProtocol.java:719)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoin
t.java:2080)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExec
utor.java:885)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:907)
at java.lang.Thread.run(Thread.java:619)
event: ERROR, subtype: null
Code for LastChunkTest:
=======================
package test;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.NoRouteToHostException;
import java.net.Socket;
import java.net.URL;
public class LastChunkTest {
private static final String ENCODING = "ISO-8859-1";
private static final String DELIMITER = "\r\n";
private URL url;
private InputStream inputStream;
private OutputStream outputStream;
private Socket socket;
public static void main(String[] args) throws Exception {
LastChunkTest test = new LastChunkTest();
test.test();
}
private void test() throws Exception {
url = new URL("http://www.seekspeak.com/CometTest";);
initConnection();
sendHeaders();
send("comet test");
// uncomment sleep to get END event instead of read error
// try {
// Thread.sleep(50);
// } catch (InterruptedException ie) {
// // do nothing
// }
sendLastChunk();
// block on read to keep app alive - server never sends response
inputStream.read();
}
private void initConnection() throws IOException {
int port = url.getPort();
port = (port < 0) ? url.getDefaultPort() : port;
try {
socket = new Socket(url.getHost(), port);
socket.setKeepAlive(true);
inputStream = socket.getInputStream();
outputStream = socket.getOutputStream();
} catch (NoRouteToHostException nrthe) {
System.out.println("host: " + url.getHost());
nrthe.printStackTrace();
}
}
private void sendHeaders() throws IOException {
String path = url.getPath();
StringBuffer outputBuffer = new StringBuffer();
outputBuffer.append("POST " + path + " HTTP/1.1" + DELIMITER);
outputBuffer.append("Host: " + url.getHost() + DELIMITER);
outputBuffer.append("User-Agent: CometTest" + DELIMITER);
outputBuffer.append("Connection: keep-alive" + DELIMITER);
outputBuffer.append("Content-Type: text/plain" + DELIMITER);
outputBuffer.append("Transfer-Encoding: chunked" + DELIMITER);
outputBuffer.append(DELIMITER);
byte[] outputBytes = outputBuffer.toString().getBytes(ENCODING);
outputStream.write(outputBytes);
outputStream.flush();
}
private void send(String chunkData) throws IOException {
byte[] chunkBytes = chunkData.getBytes(ENCODING);
String hexChunkLength = Integer.toHexString(chunkBytes.length);
StringBuffer outputBuffer = new StringBuffer();
outputBuffer.append(hexChunkLength);
outputBuffer.append(DELIMITER);
outputBuffer.append(chunkData);
outputBuffer.append(DELIMITER);
byte[] outputBytes = outputBuffer.toString().getBytes(ENCODING);
outputStream.write(outputBytes);
outputStream.flush();
}
private void sendLastChunk() throws IOException {
byte[] outputBytes = new String("0" + DELIMITER).getBytes(ENCODING);
outputStream.write(outputBytes);
outputStream.flush();
}
}
On Jan 22, 2008 9:07 AM, Filip Hanik - Dev Lists <[EMAIL PROTECTED]> wrote:
> I still don't get the END event, however, thanks for pointing it out,
> you did find a regression bug about the timeout
>
> http://svn.apache.org/viewvc?view=rev&revision=614249
>
> I've added the patch above to the proposal list for 6.0.x to fix the
> timeout issue
>
> Filip
>
>
> Peter Warren wrote:
> >> as I mentioned, the "last chunk" doesn't generate an END event, I tried
> >> it locally. of course against 6.0.x trunk.
> >>
> >
> > I played around a bit because I was definitely getting an END event
> > and found: Sending 0crlf does not generate and END event. However
> > sending 0crlfcrlf, which is what HttpURLConnection does, does generate
> > an END (or sometimes a read error - see below...) Looking at the http
> > spec, it seems like 0crlfcrlf is actually the proper way to terminate
> > the chunk body:
> >
> > Chunked-Body = *chunk
> > last-chunk
> > trailer
> > CRLF
> >
> > last-chunk = 1*("0") [ chunk-extension ] CRLF
> >
> > Am I reading that correctly?
> >
> > Note about END and read error:
> > When running both the client and the server locally (i.e. little
> > latency), sending 0crlfcrlf would sometimes generate a read error
> > (i.e. inputStream.isAvailable() > 0 would be true and then number of
> > bytes read would be < 0) and sometimes an END event.
> >
> > I tried with both sockets and HttpURLConnection and saw similar
> > behavior. However when using HttpURLConnection I could add a delay of
> > 50 millis. and guarantee that I always got an end event (see code
> > below).
> >
> > I am using the latest 6.0.x trunk updated locally today.
> >
> > Peter
> >
> > CLIENT CODE
> > ===========
> > URL url = new URL("http://www.seekspeak.com/CometTest";);
> > HttpURLConnection urlConn = (HttpURLConnection)
> > url.openConnection();
> > urlConn.setRequestMethod("POST");
> > urlConn.setChunkedStreamingMode(-1); // use default chunk length
> > urlConn.setReadTimeout(0);
> > urlConn.setDoInput(true);
> > urlConn.setDoOutput(true);
> > urlConn.connect();
> > PrintWriter out = new PrintWriter(urlConn.getOutputStream(), true);
> > out.print("test");
> > out.flush();
> > try {
> > // sleep to guarantee an END event - remove this sleep to
> > get read error
> > Thread.sleep(50);
> > } catch (InterruptedException ie) {
> > // do nothing
> > }
> > urlConn.getInputStream();
> >
> > COMET SERVLET CODE:
> > =============
> > public void event(CometEvent cometEvent) throws IOException,
> > ServletException {
> > System.out.println("event: " + cometEvent.getEventType() + ",
> > subtype: " + cometEvent.getEventSubType());
> > if (cometEvent.getEventType() == CometEvent.EventType.ERROR) {
> > cometEvent.close();
> > } else if (cometEvent.getEventType() == CometEvent.EventType.END) {
> > cometEvent.close();
> > } else if (cometEvent.getEventType() == CometEvent.EventType.READ) {
> > HttpServletRequest request = cometEvent.getHttpServletRequest();
> > InputStream inputStream = request.getInputStream();
> > byte[] buf = new byte[512];
> > do {
> > int n = inputStream.read(buf); // can throw an IOException
> > if (n > 0) {
> > System.out.println("Read " + n + " bytes: " + new
> > String(buf, 0, n) + " for session: "
> > + request.getSession(true).getId());
> > } else if (n < 0) {
> > System.out.println("read error");
> > return;
> > }
> > } while (inputStream.available() > 0);
> > }
> > }
> >
> > On Jan 21, 2008 11:53 AM, Filip Hanik - Dev Lists <[EMAIL PROTECTED]> wrote:
> >
> >> answers inline
> >>
> >> Peter Warren wrote:
> >>
> >>> First off, thanks for your responses. The contributors to this list
> >>> are extremely responsive, patient, and helpful, and I really
> >>> appreciate it!
> >>>
> >>> Hmm, in your test case did you set the HttpURLConnection to use
> >>> chunked transfers (setChunkedStreamingMode(...))? I find if I use
> >>> chunked transfers, the HttpURLConnection sends a "last chunk" message
> >>> upon reading input from the server, which generates a comet END event
> >>> on the server. If I don't use chunked transfers, no END event is
> >>> generated because no "last chunk" is sent by the client.
> >>>
> >>>
> >> as I mentioned, the "last chunk" doesn't generate an END event, I tried
> >> it locally. of course against 6.0.x trunk.
> >>
> >>> Which brings up an option I never considered: will a comet servlet
> >>> function properly with non-chunked transfer (i.e. no transfer-coding
> >>> header)? It seems to.
> >>>
> >>>
> >> yes, it can, just send a very large content-length header
> >>
> >>> Lastly, I'm still a little confused about requiring the comet event to
> >>> be closed on an END event. Doesn't this mean that tomcat comet can't
> >>> handle pipelined requests? If a client sends 2 pipelined requests, it
> >>> will send a "last chunk" to indicate the end of the first request.
> >>> This "last chunk" will generate an END event on the server, which then
> >>> requires the connection to be closed. After the comet event is closed
> >>> the server cannot send a response to the client.
> >>>
> >>>
> >> pipelined requests are not defined by the HTTP spec for POST methods,
> >> only for GET
> >> assuming the pipelining you are talking about is true pipelining :)
> >> if you just mean, next request, then yes, tomcat handles that just fine,
> >> and that is why you have to call event.close()
> >>
> >>> The END event docs indicate that pipelined request will generate an
> >>> END event: ...End will also be called when data is available and the
> >>> end of file is reached on the request input (this usually indicates
> >>> the client has pipelined a request).
> >>>
> >>>
> >> depends on what you mean by pipeline, see above
> >>
> >>
> >>> Thanks,
> >>> Peter
> >>>
> >>> On Jan 20, 2008 8:15 PM, Filip Hanik - Dev Lists <[EMAIL PROTECTED]>
> >>> wrote:
> >>>
> >>>
> >>>> now I get it. I just ran through a test case, and an END event was not
> >>>> thrown just because there was an end chunk.
> >>>> the response is very much still open at that point
> >>>>
> >>>>
> >>>> Filip
> >>>>
> >>>> Peter Warren wrote:
> >>>>
> >>>>
> >>>>> What java.net.HttpURLConnection has to do with Tomcat and comet is
> >>>>> that HttpURLConnection is Java's implementation of an http client and
> >>>>> will likely be used by people developing comet apps for Tomcat. In my
> >>>>> case, I want to use it because I can't use raw sockets on my applet
> >>>>> client due to permission problems when trying to use sockets behind a
> >>>>> proxy.
> >>>>>
> >>>>> I understand that asynchronous writes are possible, but they're not
> >>>>> when using HttpURLConnection because HttpURLConnection sends a "last
> >>>>> chunk" message when it's done with its request. "Last chunk"
> >>>>> generates a comet end event, which then requires that the connection
> >>>>> to the client be closed.
> >>>>>
> >>>>> I guess I don't understand why tomcat needs to close the connection
> >>>>> after an END event. It seems to me that the "last chunk" message from
> >>>>> the client simply indicates that the client is done sending its
> >>>>> request. Why does the server need to close the connection when the
> >>>>> client finishes its request?
> >>>>>
> >>>>> Peter
> >>>>>
> >>>>> On Jan 19, 2008 6:01 PM, Filip Hanik - Dev Lists <[EMAIL PROTECTED]>
> >>>>> wrote:
> >>>>>
> >>>>>
> >>>>>
> >>>>>> I'm not sure what HttpURLConnection has to do with Tomcat or comet.
> >>>>>> and yes, asynchronous writes are possible, just not after the END or
> >>>>>> ERROR events have been issued
> >>>>>>
> >>>>>> Filip
> >>>>>>
> >>>>>>
> >>>>>> Peter Warren wrote:
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>> Does that mean that HttpURLConnection cannot be used for comet
> >>>>>>> requests with asynchronous (i.e. delayed) responses?
> >>>>>>>
> >>>>>>> It would seem so to me since HttpURLConnection always sends an END
> >>>>>>> message before reading from the server and since the server can no
> >>>>>>> longer write to the client after closing the comet event. Am I
> >>>>>>> missing something? Is there a way to write to the client after the
> >>>>>>> comet event is closed?
> >>>>>>>
> >>>>>>> Would you consider it a bug that HttpURLConnection is implemented
> >>>>>>> that way?
> >>>>>>>
> >>>>>>> Peter
> >>>>>>>
> >>>>>>> On Jan 18, 2008 9:21 PM, Filip Hanik - Dev Lists <[EMAIL PROTECTED]>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>> during end and error, you MUST close the Comet event
> >>>>>>>>
> >>>>>>>> Filip
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Peter Warren wrote:
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> What do I do to make the END event stop repeating? I don't want to
> >>>>>>>>> close the CometEvent yet because the server is waiting for data to
> >>>>>>>>> send to the client. If I don't close the comet event, the END event
> >>>>>>>>> repeats incessantly.
> >>>>>>>>>
> >>>>>>>>> I'm using an unsigned applet as a comet client. To accommodate
> >>>>>>>>> proxies, I've had to change my comet client to use HttpURLConnection
> >>>>>>>>> instead of Sockets. (Accessing ProxySelector from an applet to
> >>>>>>>>> create
> >>>>>>>>> a socket with a proxy generates an AccessControlException.)
> >>>>>>>>>
> >>>>>>>>> HttpURLConnection unfortunately sends a 0crlf when its input stream
> >>>>>>>>> is
> >>>>>>>>> retrieved for reading. This generates a Comet END event. Short of
> >>>>>>>>> closing the comet event, how can I make the server stop notifying me
> >>>>>>>>> of END events? I can't close the comet event because I want to hold
> >>>>>>>>> onto the comet output stream for use later to send data to the
> >>>>>>>>> client.
> >>>>>>>>>
> >>>>>>>>> >From the comet docs:
> >>>>>>>>> EventType.END: End may be called to end the processing of the
> >>>>>>>>> request.
> >>>>>>>>> Fields that have been initialized in the begin method should be
> >>>>>>>>> reset.
> >>>>>>>>> After this event has been processed, the request and response
> >>>>>>>>> objects,
> >>>>>>>>> as well as all their dependent objects will be recycled and used to
> >>>>>>>>> process other requests. End will also be called when data is
> >>>>>>>>> available
> >>>>>>>>> and the end of file is reached on the request input (this usually
> >>>>>>>>> indicates the client has pipelined a request).
> >>>>>>>>>
> >>>>>>>>> This seems to indicate that even if I could get the END event to go
> >>>>>>>>> away quietly, the comet event's output stream might no longer be
> >>>>>>>>> usable anyway.
> >>>>>>>>>
> >>>>>>>>> It seems to me I have 3 options:
> >>>>>>>>> 1) figure out how to make the comet END event stop repeating and
> >>>>>>>>> hope
> >>>>>>>>> it's output stream still works
> >>>>>>>>> 2) figure out how to keep HttpURLConnection from sending 0crlf
> >>>>>>>>> (don't
> >>>>>>>>> know if that can be done)
> >>>>>>>>> 2) use sockets with ProxySelector (which requires signing my applet
> >>>>>>>>> and getting users to grant it privileges)
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> Peter
> >>>>>>>>>
> >>>>>>>>> ---------------------------------------------------------------------
> >>>>>>>>> To start a new topic, e-mail: users@tomcat.apache.org
> >>>>>>>>> To unsubscribe, e-mail: [EMAIL PROTECTED]
> >>>>>>>>> For additional commands, e-mail: [EMAIL PROTECTED]
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>> ---------------------------------------------------------------------
> >>>>>>>> To start a new topic, e-mail: users@tomcat.apache.org
> >>>>>>>> To unsubscribe, e-mail: [EMAIL PROTECTED]
> >>>>>>>> For additional commands, e-mail: [EMAIL PROTECTED]
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>> ---------------------------------------------------------------------
> >>>>>>> To start a new topic, e-mail: users@tomcat.apache.org
> >>>>>>> To unsubscribe, e-mail: [EMAIL PROTECTED]
> >>>>>>> For additional commands, e-mail: [EMAIL PROTECTED]
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>> ---------------------------------------------------------------------
> >>>>>> To start a new topic, e-mail: users@tomcat.apache.org
> >>>>>> To unsubscribe, e-mail: [EMAIL PROTECTED]
> >>>>>> For additional commands, e-mail: [EMAIL PROTECTED]
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>> ---------------------------------------------------------------------
> >>>>> To start a new topic, e-mail: users@tomcat.apache.org
> >>>>> To unsubscribe, e-mail: [EMAIL PROTECTED]
> >>>>> For additional commands, e-mail: [EMAIL PROTECTED]
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>> ---------------------------------------------------------------------
> >>>> To start a new topic, e-mail: users@tomcat.apache.org
> >>>> To unsubscribe, e-mail: [EMAIL PROTECTED]
> >>>> For additional commands, e-mail: [EMAIL PROTECTED]
> >>>>
> >>>>
> >>>>
> >>>>
> >>> ---------------------------------------------------------------------
> >>> To start a new topic, e-mail: users@tomcat.apache.org
> >>> To unsubscribe, e-mail: [EMAIL PROTECTED]
> >>> For additional commands, e-mail: [EMAIL PROTECTED]
> >>>
> >>>
> >>>
> >>>
> >>>
> >> ---------------------------------------------------------------------
> >> To start a new topic, e-mail: users@tomcat.apache.org
> >> To unsubscribe, e-mail: [EMAIL PROTECTED]
> >> For additional commands, e-mail: [EMAIL PROTECTED]
> >>
> >>
> >>
> >
> > ---------------------------------------------------------------------
> > To start a new topic, e-mail: users@tomcat.apache.org
> > To unsubscribe, e-mail: [EMAIL PROTECTED]
> > For additional commands, e-mail: [EMAIL PROTECTED]
> >
> >
> >
> >
>
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
