Re: Tomcat 6.0.24 - Should send body As Chunked Response but sometimes sends it back Un-Chunked
On Thu, Nov 25, 2010 at 6:48 PM, Pid wrote:
> On 25/11/2010 17:49, declan harrison wrote:
>>> > So far you've shown me that you wrap the output stream, and how the
>>> > write method works.
>>> >
>>> > You haven't actually posted the part of the code where you write your
>>> > bit of the data to the wrapped output stream.
>>> >
>>> >
>>> > p
>> I just call wrapper.write(buf, 0, bug.lenght). Are you looking more
>> source code here?
>
> You are reporting a problem with the response.
>
> You are writing to the output stream, and maybe setting headers on the
> response object.
>
> If you can post each bit of code that interacts with the response object
> we can stop guessing what's wrong and start trying to resolve it.
>
>
> p
>
Apologises about not posting enough of the code, see below for the
methods that sets the respos headers etc
In the method writeResponseHdrs (shown below) we end up reject the
request as bit rate too low and end up executing this code
Log.AdaptVideoError("Video bitrate
["+inputBitRate+"] is
not within range", "Will not encode this video.");
throw new InitialisationException("Video bitrate
["+inputBitRate+"] is not within range. Will not encode this video");
This exception is caught higher up the stack in doPost method which we
then set the status with the following
bufInStream = new
AcceleratorInputStream(request.getInputStream(), 20480);
bufInStream.setHeaderRead();
bufOutStream = new
AcceleratorOutputStream(response.getOutputStream(),20480);
try {
converter = createVideoAdapter(request,
response,
bufInStream,bufOutStream,
request.getContentType(), true,
qualityPolicy);
} catch (Exception ex) {
// caught the exception ok, make sure we have
commited the response
response.sendError(HttpServletResponse.SC_INTERNAL_SERVER_ERROR );
== status code set here and sent in do Post
return;
}
So this is the request that I terminate on the client end before the
respone is actually sent by Tomcat, Im not exactly sure were in Tomcat
the code is when I terminate client request. If I terminate at the
correct point then the next request that re-uses that thread is wrong.
/**
* Write the initial response to the client.
*
* Must write headers before ressponse is committed.
* @throws InitialisationException
* @throws IOException
*
*/
private void writeResponseHdrs() throws InitialisationException,
IOException {
//
// Lets iterate thru the hdrs in the response and
if (responseSet == false) {
setResponseHeaders();
// lets set flag to indicate that hdr have been set
responseSet = true;
//inputStream.setHeaderRead();
outputStream.setHeaderWritten();
// set the status code as well
//
//
if (validateVideoBitate(inputBitRate) == false ) {
//Bitrate check failed - return duration to VR to
facilitate JIT
if (writeResponseHdrs == true) {
streamDuration = mIContainer.getDuration();
response.setContentType( mimeType );
if (metaDuration > 0.0d) {
response.addHeader("x-duration",
String.valueOf(metaDuration));
Log.AdaptVideo("Adding video duration
header on
BR failure [x-duration: "+
String.valueOf(metaDuration)+"]
extracted from onMetaDataTag.");
} else {
// in microseconds
response.addHeader("x-duration",
String.valueOf(streamDuration/100));
Log.AdaptVideo("Adding video duration
header on
BR failure [x-duration: "+
String.valueOf(streamDuration/100)+"] extracted from container
API.");
}
outputStream.setHeaderWritten();
}
Log.AdaptVideoError("Video bitrate
["+inputBitRate+"] is
not within range", "Will not encode this video.");
throw new InitialisationException("Video bitrate
["+inputBitRate+"] is not within range. Will not encode this video");
}
}
}
private void setR
Re: Tomcat 6.0.24 - Should send body As Chunked Response but sometimes sends it back Un-Chunked
On 25/11/2010 17:49, declan harrison wrote: >> > So far you've shown me that you wrap the output stream, and how the >> > write method works. >> > >> > You haven't actually posted the part of the code where you write your >> > bit of the data to the wrapped output stream. >> > >> > >> > p > I just call wrapper.write(buf, 0, bug.lenght). Are you looking more > source code here? You are reporting a problem with the response. You are writing to the output stream, and maybe setting headers on the response object. If you can post each bit of code that interacts with the response object we can stop guessing what's wrong and start trying to resolve it. p 0x62590808.asc Description: application/pgp-keys signature.asc Description: OpenPGP digital signature
Re: Tomcat 6.0.24 - Should send body As Chunked Response but sometimes sends it back Un-Chunked
On Thu, Nov 25, 2010 at 11:49 AM, Pid wrote:
> On 25/11/2010 11:14, declan harrison wrote:
>> On Thu, Nov 25, 2010 at 10:37 AM, Pid wrote:
>>> On 25/11/2010 10:22, declan harrison wrote:
> ... and how are you writing to the BufferedOutputStream?
>>
So I end up calling this method for most of the writes that are
performed for binary IO.
@Override
public synchronized void write(byte[] b, int off, int len)
throws IOException {
long bTime = System.currentTimeMillis();
super.write(b, off, len);
long aTime = System.currentTimeMillis();
timeDelta += aTime-bTime;
++numWrites;
updateSNMPValue(bWrote);
}
>>>
>>> Sorry, how does this answer my question?
>>>
>>>
>> I must have misunderstood your question. What do you mean exactly by
>> the question "and how are you writing to the BufferedOutputStream?"
>>
>> I call the write method above when writing out the binary data in the
>> response body. Is this the information you require or am I missing
>> something? Apoligises if Im being silly on this.
>
> So far you've shown me that you wrap the output stream, and how the
> write method works.
>
> You haven't actually posted the part of the code where you write your
> bit of the data to the wrapped output stream.
>
>
> p
I just call wrapper.write(buf, 0, bug.lenght). Are you looking more
source code here?
>
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat 6.0.24 - Should send body As Chunked Response but sometimes sends it back Un-Chunked
On Thu, Nov 25, 2010 at 3:08 PM, Caldarale, Charles R wrote: >> From: declan harrison [mailto:harrison.dec...@gmail.com] >> Subject: Re: Tomcat 6.0.24 - Should send body As Chunked Response but >> sometimes sends it back Un-Chunked > >> it does look like it to me that state is leaking between >> these two request on this same thread. > > That hints of a ThreadLocal being used somewhere in your code (or a 3rd-party > library). That's usually not a good idea in a pooled environment. > Must be in 3rd party code as I dont use any ThreadLocal data explicitly myself. >> Is there a way of obliterating a thread on Tomcat > > Not that I'm aware of. You could temporarily set clearReferencesThreadLocals > to true in your element to see if that takes care of the problem, > but you probably don't want to do that in production (not really thread-safe). Yea not sure. I was wondering it might be worth trying to use the APR Connector and see if the issue is the present there also or hopefully not. you think its worth a shot. Im also looking at trying to produce a simple servlet where I can reproduce this problem and send it to the mailing list. I'm hopeful I can get such a simple servlet - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Tomcat 6.0.24 - Should send body As Chunked Response but sometimes sends it back Un-Chunked
> From: declan harrison [mailto:harrison.dec...@gmail.com] > Subject: Re: Tomcat 6.0.24 - Should send body As Chunked Response but > sometimes sends it back Un-Chunked > it does look like it to me that state is leaking between > these two request on this same thread. That hints of a ThreadLocal being used somewhere in your code (or a 3rd-party library). That's usually not a good idea in a pooled environment. > Is there a way of obliterating a thread on Tomcat Not that I'm aware of. You could temporarily set clearReferencesThreadLocals to true in your element to see if that takes care of the problem, but you probably don't want to do that in production (not really thread-safe). - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat 6.0.24 - Should send body As Chunked Response but sometimes sends it back Un-Chunked
On 25/11/2010 11:14, declan harrison wrote:
> On Thu, Nov 25, 2010 at 10:37 AM, Pid wrote:
>> On 25/11/2010 10:22, declan harrison wrote:
... and how are you writing to the BufferedOutputStream?
>
>>> So I end up calling this method for most of the writes that are
>>> performed for binary IO.
>>> @Override
>>> public synchronized void write(byte[] b, int off, int len)
>>> throws IOException {
>>>
>>> long bTime = System.currentTimeMillis();
>>> super.write(b, off, len);
>>> long aTime = System.currentTimeMillis();
>>> timeDelta += aTime-bTime;
>>> ++numWrites;
>>>
>>> updateSNMPValue(bWrote);
>>> }
>>>
>>
>> Sorry, how does this answer my question?
>>
>>
> I must have misunderstood your question. What do you mean exactly by
> the question "and how are you writing to the BufferedOutputStream?"
>
> I call the write method above when writing out the binary data in the
> response body. Is this the information you require or am I missing
> something? Apoligises if Im being silly on this.
So far you've shown me that you wrap the output stream, and how the
write method works.
You haven't actually posted the part of the code where you write your
bit of the data to the wrapped output stream.
p
0x62590808.asc
Description: application/pgp-keys
signature.asc
Description: OpenPGP digital signature
Re: Tomcat 6.0.24 - Should send body As Chunked Response but sometimes sends it back Un-Chunked
On Thu, Nov 25, 2010 at 10:37 AM, Pid wrote:
> On 25/11/2010 10:22, declan harrison wrote:
>>> ... and how are you writing to the BufferedOutputStream?
>>> >
>> So I end up calling this method for most of the writes that are
>> performed for binary IO.
>> @Override
>> public synchronized void write(byte[] b, int off, int len)
>> throws IOException {
>>
>> long bTime = System.currentTimeMillis();
>> super.write(b, off, len);
>> long aTime = System.currentTimeMillis();
>> timeDelta += aTime-bTime;
>> ++numWrites;
>>
>> updateSNMPValue(bWrote);
>> }
>>
>
> Sorry, how does this answer my question?
>
>
I must have misunderstood your question. What do you mean exactly by
the question "and how are you writing to the BufferedOutputStream?"
I call the write method above when writing out the binary data in the
response body. Is this the information you require or am I missing
something? Apoligises if Im being silly on this.
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat 6.0.24 - Should send body As Chunked Response but sometimes sends it back Un-Chunked
On 25/11/2010 10:22, declan harrison wrote:
>> ... and how are you writing to the BufferedOutputStream?
>> >
> So I end up calling this method for most of the writes that are
> performed for binary IO.
> @Override
> public synchronized void write(byte[] b, int off, int len)
> throws IOException {
>
> long bTime = System.currentTimeMillis();
> super.write(b, off, len);
> long aTime = System.currentTimeMillis();
> timeDelta += aTime-bTime;
> ++numWrites;
>
> updateSNMPValue(bWrote);
> }
>
Sorry, how does this answer my question?
p
0x62590808.asc
Description: application/pgp-keys
signature.asc
Description: OpenPGP digital signature
Re: Tomcat 6.0.24 - Should send body As Chunked Response but sometimes sends it back Un-Chunked
On Wed, Nov 24, 2010 at 10:39 PM, Caldarale, Charles R wrote: >> From: declan harrison [mailto:harrison.dec...@gmail.com] >> Subject: Re: Tomcat 6.0.24 - Should send body As Chunked Response but >> sometimes sends it back Un-Chunked > >> I meant I have local variable that reference the response >> output stream. > > Ok, you're off the hook on that one. Anything stored in the session that > might allow intermingling of requests or responses? > Chuck I have reviewed the code this morning and I canny see anything that would allow the intermingling of the request or response between separate HTTP requests but it does look like it to me that state is leaking between these two request on this same thread. Is there a way of obliterating a thread on Tomcat, this way I could verify that this indeed is the issue. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat 6.0.24 - Should send body As Chunked Response but sometimes sends it back Un-Chunked
On Thu, Nov 25, 2010 at 9:45 AM, Pid wrote:
> On 24/11/2010 22:11, declan harrison wrote:
>>> How are you writing to the output of the response?
>>> >
>> Im wrapping the response ServletOutputStream in a BufferedOutputStream;
>> Something like this bufOutStream = new
>> BufferedOutputStream(response.getOutputStream(),20480);
>
> ... and how are you writing to the BufferedOutputStream?
>
So I end up calling this method for most of the writes that are
performed for binary IO.
@Override
public synchronized void write(byte[] b, int off, int len)
throws IOException {
long bTime = System.currentTimeMillis();
super.write(b, off, len);
long aTime = System.currentTimeMillis();
timeDelta += aTime-bTime;
++numWrites;
updateSNMPValue(bWrote);
}
Im not clear when I call response.addHeader and response.sendError,
what or if the BufferedOutput stream IO is called by Tomcat Coyote.
When I debug the Web App I see that BufferedOutputStream underlying
OutputStream (protected) is in fact a CoyoteOutputStream (I can access
this object) and it contains a protected object called
org.apache.catalina.connector.OutputBuffer. I would dearly like to
get at this objects data but I cannot as its protected, this contains
the headers and details about the stream and its state.
Is there a way to dump out this object OutputBuffer details. Is there
any debug logs I can enable here?
Thanks
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat 6.0.24 - Should send body As Chunked Response but sometimes sends it back Un-Chunked
On 24/11/2010 22:11, declan harrison wrote: >> How are you writing to the output of the response? >> > > Im wrapping the response ServletOutputStream in a BufferedOutputStream; > Something like this bufOutStream = new > BufferedOutputStream(response.getOutputStream(),20480); ... and how are you writing to the BufferedOutputStream? p 0x62590808.asc Description: application/pgp-keys signature.asc Description: OpenPGP digital signature
RE: Tomcat 6.0.24 - Should send body As Chunked Response but sometimes sends it back Un-Chunked
> From: declan harrison [mailto:harrison.dec...@gmail.com] > Subject: Re: Tomcat 6.0.24 - Should send body As Chunked Response but > sometimes sends it back Un-Chunked > I meant I have local variable that reference the response > output stream. Ok, you're off the hook on that one. Anything stored in the session that might allow intermingling of requests or responses? - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat 6.0.24 - Should send body As Chunked Response but sometimes sends it back Un-Chunked
On Wed, Nov 24, 2010 at 10:23 PM, Caldarale, Charles R wrote: > From: declan harrison [mailto:harrison.dec...@gmail.com] >> Subject: Re: Tomcat 6.0.24 - Should send body As Chunked Response but >> sometimes sends it back Un-Chunked > >> > Do you have any part of the response stored as an >> > instance field in your servlet? > >> Yes I have a reference to the HttpServletResponse, >> response.getOutputStream(), which I close when Im done. > > Fatal error - don't ever do that. Multiple threads running through your > servlet will walk over each other. It's fine to have local variables > pointing to objects related to a request or response, but having instance or > static variables do so is guaranteed to cause problems. Remember that each > servlet (or filter) is instantiated *once* for the lifetime of the webapp, > not once per request. > > - Chuck Hi Chuck Sorry for my misleading statement I meant I have local variable that reference the response output stream. I dont have a servlet attribute that holds a reference to the response stream > > > THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY > MATERIAL and is thus for use only by the intended recipient. If you received > this in error, please contact the sender and delete the e-mail and its > attachments from all computers. > > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat 6.0.24 - Should send body As Chunked Response but sometimes sends it back Un-Chunked
On Wed, Nov 24, 2010 at 7:59 PM, Christopher Schultz wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Pid, > > On 11/24/2010 2:32 PM, Pid * wrote: >> That sounds like the thread is only being reset properly after it's >> been used a second time. > > +1 > > This can probably be replicated in a test by having a dummy servlet > simply throw a ClientAbortException (or whatever it is that gets thrown > when you try to write to a socket whose client has already hung up the > phone). > > Declan, could you try to build a simple servlet that replicates these > conditions and post it? That will go a long way towards getting this > investigated and, if it's a problem with Tomcat, ultimately fixed. > Hi Christopher Yes, I can look to build a servlet to do this; to throw an exception when performing IO on the ServletoutputStream thats been tore down. Hopefully will get back tomorrow > - -chris > -BEGIN PGP SIGNATURE- > Version: GnuPG v1.4.10 (MingW32) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iEYEARECAAYFAkztbrsACgkQ9CaO5/Lv0PAH2gCdH/u7XLeA1Mh6uORcU7PlJQj1 > lIQAn1mhJnnFc2Mo6lH5r0dYU90X4LRK > =S+Ki > -END PGP SIGNATURE- > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Tomcat 6.0.24 - Should send body As Chunked Response but sometimes sends it back Un-Chunked
From: declan harrison [mailto:harrison.dec...@gmail.com] > Subject: Re: Tomcat 6.0.24 - Should send body As Chunked Response but > sometimes sends it back Un-Chunked > > Do you have any part of the response stored as an > > instance field in your servlet? > Yes I have a reference to the HttpServletResponse, > response.getOutputStream(), which I close when Im done. Fatal error - don't ever do that. Multiple threads running through your servlet will walk over each other. It's fine to have local variables pointing to objects related to a request or response, but having instance or static variables do so is guaranteed to cause problems. Remember that each servlet (or filter) is instantiated *once* for the lifetime of the webapp, not once per request. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat 6.0.24 - Should send body As Chunked Response but sometimes sends it back Un-Chunked
On Wed, Nov 24, 2010 at 7:32 PM, Pid * wrote: > On 24 Nov 2010, at 18:07, declan harrison wrote: > >> Hi >> >> This is my first post to tomcat user mailing list. > > Hello. Hi there > >> I'm using Tomcat 6.0.24 on Linux RHELS release 5.3 (Tikanga) on a 64 bit OS. > > Which JVM version? java version "1.6.0_18" Java(TM) SE Runtime Environment (build 1.6.0_18-b07) Java HotSpot(TM) 64-Bit Server VM (build 16.0-b13, mixed mode) > >> I have deployed a Web application within Tomcat. The Web App in the >> main sends dynamically created video content to a client over the >> HTTP1.1 Blocking Connector. > > How are you writing to the output of the response? > Im wrapping the response ServletOutputStream in a BufferedOutputStream; Something like this bufOutStream = new BufferedOutputStream(response.getOutputStream(),20480); >> I have configured Tomcat to use the Blocking IO HTTP Connector. Main >> Configuration parameters are the following maxThreads="1000" >> minSpareThreads="300" >> >> The use case is the following. HTTP Client uses HTTP Post to send >> video content to a Web App in Tomcat. Tomcat Web App Dynamically >> re-codes this video content and sends this video payload in the body >> of the HTTP Response (response to HTTP Post) to the client. >> >> As the Web App doesn't know the final size of the video content it >> sends so it does not add the HTTP "Content-Length" header in the >> HttpServletResponse. The Tomcat HTTP Connector (I presume) correctly >> knows to send this content as a HTTP Chunked response, i.e. >> Transfer-Encoding: Chunked". >> >> So this work well nearly all of the time, i.e. correct headers are >> sent and all the data is sent as chunked response. However when I am >> putting a load through the system e.g 40 or 50 concurrent threads >> processing content and I make another single HTTP request to the Web >> App that is normally rejected with a 500. > > What us the cause, and is there an exception stacktrace in the logs? It is expected for my Web App to reject this request with HTTP status code 500. The input request content is invalid. So typically the Web App will log the problem with the input content and then call response.sendError(HttpServletResponse.SC_INTERNAL_SERVER_ERROR ); I dont have an error code setup in the web.xml file. After having setup some HTTP response headers and then return from the doPost method. There is no exception backtrace as such, logged by tomcat as far as I can see. do you think that Tomcat itself is logging a stacktrace somewhere. Theres nothing in my log file apart from the expected warning/error message that the Web App log explicitly. > >> However before the Tomcat >> thread finalizes it response to this specific request I terminate the >> client and its request (ctrl-c), then the next time this same Tomcat >> thread is re-used it will send the video in the HTTP response body as >> Un-Chunked and not Chunked as expected but the HTTP status code is 200 > > How are you determining that it is the same thread? So in my Web App log file i log the client IP address and Port. So for the client request which I terminate manually I perform this from a different IP address than the one were all the other requests are made from. The log file also logs the ThreadId of each log event e.g. Nov 22 17:54:19 hostname [ProcessName][1001][26348][tomcat-exec-243] INFO Bla Bla... Here you can see "tomcat-exec-243" is the thread Id. So then the next time a log event is performed that matches this threadId I see that it fails. > >> as expected. Also the "Content-Length" header is missing on this >> response and all the other HTTP headers that the Web App inserted are >> not sent. This then causes problems for the HTTP Client requesting >> said content. >> >> However the next time after this that we re-use this thread (3rd time >> used) then its back to working normally again sending HTTP Chunked >> response as exepcted. > > That sounds like the thread is only being reset properly after it's > been used a second time. > Yes that correct. It appears to me that the state of the output stream in the thread is all setup to send a non chunked response as it would with the 500 error it didnt send and so sends the video response non chunked and its only after that does it resset its state. Does each thread maintain state between different HTTP requests. I also tried increasing linger time on each socket. So I tried 5 seconds and right enough it took 5 or more seconds before Tomcat re-used this thread but the bad result was the same. >> So the normal response is like this. Note chunked response and chunk >> size is "2000" hex about 8K. >> >> HTTP/1.1 200 OK >> Server: Apache-Coyote/1.1 >> x-my-session-id: 1000128946853721720696116 >> Content-Type: video/x-flv >> Transfer-Encoding: chunked >> Date: Thu, 11 Nov 2010 09:42:17 GMT >> >> 2000 >> onMetaData >> totalduration >> canseekontime >> bytelength >> >> The problems response looks this this. Note
Re: Tomcat 6.0.24 - Should send body As Chunked Response but sometimes sends it back Un-Chunked
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Pid, On 11/24/2010 2:32 PM, Pid * wrote: > That sounds like the thread is only being reset properly after it's > been used a second time. +1 This can probably be replicated in a test by having a dummy servlet simply throw a ClientAbortException (or whatever it is that gets thrown when you try to write to a socket whose client has already hung up the phone). Declan, could you try to build a simple servlet that replicates these conditions and post it? That will go a long way towards getting this investigated and, if it's a problem with Tomcat, ultimately fixed. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkztbrsACgkQ9CaO5/Lv0PAH2gCdH/u7XLeA1Mh6uORcU7PlJQj1 lIQAn1mhJnnFc2Mo6lH5r0dYU90X4LRK =S+Ki -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat 6.0.24 - Should send body As Chunked Response but sometimes sends it back Un-Chunked
On 24 Nov 2010, at 18:07, declan harrison wrote: > Hi > > This is my first post to tomcat user mailing list. Hello. > I'm using Tomcat 6.0.24 on Linux RHELS release 5.3 (Tikanga) on a 64 bit OS. Which JVM version? > I have deployed a Web application within Tomcat. The Web App in the > main sends dynamically created video content to a client over the > HTTP1.1 Blocking Connector. How are you writing to the output of the response? > I have configured Tomcat to use the Blocking IO HTTP Connector. Main > Configuration parameters are the following maxThreads="1000" > minSpareThreads="300" > > The use case is the following. HTTP Client uses HTTP Post to send > video content to a Web App in Tomcat. Tomcat Web App Dynamically > re-codes this video content and sends this video payload in the body > of the HTTP Response (response to HTTP Post) to the client. > > As the Web App doesn't know the final size of the video content it > sends so it does not add the HTTP "Content-Length" header in the > HttpServletResponse. The Tomcat HTTP Connector (I presume) correctly > knows to send this content as a HTTP Chunked response, i.e. > Transfer-Encoding: Chunked". > > So this work well nearly all of the time, i.e. correct headers are > sent and all the data is sent as chunked response. However when I am > putting a load through the system e.g 40 or 50 concurrent threads > processing content and I make another single HTTP request to the Web > App that is normally rejected with a 500. What us the cause, and is there an exception stacktrace in the logs? > However before the Tomcat > thread finalizes it response to this specific request I terminate the > client and its request (ctrl-c), then the next time this same Tomcat > thread is re-used it will send the video in the HTTP response body as > Un-Chunked and not Chunked as expected but the HTTP status code is 200 How are you determining that it is the same thread? > as expected. Also the "Content-Length" header is missing on this > response and all the other HTTP headers that the Web App inserted are > not sent. This then causes problems for the HTTP Client requesting > said content. > > However the next time after this that we re-use this thread (3rd time > used) then its back to working normally again sending HTTP Chunked > response as exepcted. That sounds like the thread is only being reset properly after it's been used a second time. > So the normal response is like this. Note chunked response and chunk > size is "2000" hex about 8K. > > HTTP/1.1 200 OK > Server: Apache-Coyote/1.1 > x-my-session-id: 1000128946853721720696116 > Content-Type: video/x-flv > Transfer-Encoding: chunked > Date: Thu, 11 Nov 2010 09:42:17 GMT > > 2000 > onMetaData > totalduration > canseekontime > bytelength > > The problems response looks this this. Note no Content-length header > or Chunked response. > HTTP/1.1 200 OK > Server: Apache-Coyote/1.1 > Date: Wed, 10 Nov 2010 14:32:25 GMT > Connection: close > > FLV... > > To replicate this issue is very much dependant on when I terminate the > client request. If I terminate too late or too early I dont see this > issue. It some sort of race condition. Do you still see a 500 when you terminate, if so is it the same stacktrace? Do you have any part of the response stored as an instance field in your servlet? p > I upgraded to Tomcat 6.0.29 as I had seen some security issues > relating to Transfer-Encodings, here > http://tomcat.apache.org/security-6.html. However it didnt make any > difference. > > I have also read the bug list and cannot see this same issue. > > Any ideas what the issue is here and is there anything I can do to mitigate > it? > > Thanks > Declan > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
