Re: Tomcat 9.0 async read becomes blocking with chunked transfer-encoding
Thank you Mark Andrew > On Oct 8, 2021, at 1:44 AM, Mark Thomas wrote: > > On 07/10/2021 22:23, Javateck wrote: >> Hi Mark, >> Just wondering whether we have a radar to track this, will it be in release >> notes for next release? > > The fix is in 9.0.54 and is listed in the changelog. > > Mark > >> Thanks, >> Andrew On Sep 27, 2021, at 8:54 AM, Mark Thomas wrote: >>> >>> On 27/09/2021 15:55, Mark Thomas wrote: > On 27/09/2021 09:08, Goldengate liu wrote: > Hi Mark, > >I’m uploading some test files Thanks for the test case. I'm looking at this now. >>> >>> Bug found and fixed. >>> >>> One thing to note is that with chunked encoding it is possible for you to >>> see isReady() return true only for the subsequent read to return 0 bytes. >>> This happens when just (or only part of) the chunked header is available. >>> >>> The sample code you provided handled this correctly. >>> >>> The fix will be in the October release round. The release process for that >>> should hopefully start later today. >>> >>> Mark >>> >>> - >>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >>> For additional commands, e-mail: users-h...@tomcat.apache.org >>> >> - >> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >> For additional commands, e-mail: users-h...@tomcat.apache.org > > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat 9.0 async read becomes blocking with chunked transfer-encoding
On 07/10/2021 22:23, Javateck wrote: Hi Mark, Just wondering whether we have a radar to track this, will it be in release notes for next release? The fix is in 9.0.54 and is listed in the changelog. Mark Thanks, Andrew On Sep 27, 2021, at 8:54 AM, Mark Thomas wrote: On 27/09/2021 15:55, Mark Thomas wrote: On 27/09/2021 09:08, Goldengate liu wrote: Hi Mark, I’m uploading some test files Thanks for the test case. I'm looking at this now. Bug found and fixed. One thing to note is that with chunked encoding it is possible for you to see isReady() return true only for the subsequent read to return 0 bytes. This happens when just (or only part of) the chunked header is available. The sample code you provided handled this correctly. The fix will be in the October release round. The release process for that should hopefully start later today. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat 9.0 async read becomes blocking with chunked transfer-encoding
Hi Mark, Just wondering whether we have a radar to track this, will it be in release notes for next release? Thanks, Andrew > On Sep 27, 2021, at 8:54 AM, Mark Thomas wrote: > > On 27/09/2021 15:55, Mark Thomas wrote: >>> On 27/09/2021 09:08, Goldengate liu wrote: >>> Hi Mark, >>> >>>I’m uploading some test files >> Thanks for the test case. I'm looking at this now. > > Bug found and fixed. > > One thing to note is that with chunked encoding it is possible for you to see > isReady() return true only for the subsequent read to return 0 bytes. This > happens when just (or only part of) the chunked header is available. > > The sample code you provided handled this correctly. > > The fix will be in the October release round. The release process for that > should hopefully start later today. > > Mark > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat 9.0 async read becomes blocking with chunked transfer-encoding
Mark, Thank you very much for the quick fix Andrew > On Sep 27, 2021, at 8:54 AM, Mark Thomas wrote: > > On 27/09/2021 15:55, Mark Thomas wrote: >>> On 27/09/2021 09:08, Goldengate liu wrote: >>> Hi Mark, >>> >>>I’m uploading some test files >> Thanks for the test case. I'm looking at this now. > > Bug found and fixed. > > One thing to note is that with chunked encoding it is possible for you to see > isReady() return true only for the subsequent read to return 0 bytes. This > happens when just (or only part of) the chunked header is available. > > The sample code you provided handled this correctly. > > The fix will be in the October release round. The release process for that > should hopefully start later today. > > Mark > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat 9.0 async read becomes blocking with chunked transfer-encoding
On 27/09/2021 15:55, Mark Thomas wrote: On 27/09/2021 09:08, Goldengate liu wrote: Hi Mark, I’m uploading some test files Thanks for the test case. I'm looking at this now. Bug found and fixed. One thing to note is that with chunked encoding it is possible for you to see isReady() return true only for the subsequent read to return 0 bytes. This happens when just (or only part of) the chunked header is available. The sample code you provided handled this correctly. The fix will be in the October release round. The release process for that should hopefully start later today. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat 9.0 async read becomes blocking with chunked transfer-encoding
On 27/09/2021 09:08, Goldengate liu wrote:
Hi Mark,
I’m uploading some test files
Thanks for the test case. I'm looking at this now.
Mark
Below are the steps:
1. compile GetPayloadServlet.java and put it to
webapps/test/WEB-INF/classes/test/
2. put web.xml to webapps/test/WEB-INF/
3. start tomcat (9.0.19), I tried with 9.0.53, I got the same result
4. use HttpPostTest(I’m using Apache httpclient-4.5.1.jar,
httpcore-4.4.4.jar), the test is sending data with chunked
transfer-encoding, one step here
4.1. put a debug point before streamWrite at flushBuffer method in
org.apache.http.impl.io.SessionOutputBufferImpl, the purpose is to pause
flushing to the socket
*private* *void* flushBuffer() *throws* IOException {
*final* *int* len = *this*.buffer.length();
*if* (len > 0) {
streamWrite(*this*.buffer.buffer(), 0, len);
*this*.buffer.clear();
*this*.metrics.incrementBytesTransferred(len);
}
}
5. put a debug point on GetPayloadServlet
@Override
public void onDataAvailable() throws IOException {
byte buffer[] = new byte[BUFFER_SIZE];
while(inputStream.isReady()) {// when isReady returns true,
inputStream.read(buffer) is actually blocked with chunked encoding
int read = inputStream.read(buffer);
if (read < 0) {
break;
}
bos.write(buffer, 0, read);
}
}
6. we can see once SessionOutputBufferImpl.flushBuffer flushes the
data,inputStream.read(buffer) inside GetPayloadServlet will be
un-blocked with read
Or am I doing something wrong? this is a basic use case.
Thanks,
Andrew
On Sep 22, 2021, at 1:14 AM, Mark Thomas > wrote:
On 22/09/2021 08:22, Goldengate liu wrote:
Hi Chris,
Servlet 3.1 spec defines that ServletInputStream can be used to
read as non-blocking way as long as there is data ready locally by
calling isReady method and check the ready condition before calling
read, and read should throw IllegalStateException if called by
caller when data is not ready
The above only applies if the servlet is in async mode. Is it?
correct, when it’s running as async mode
OK. You are going to need to provide the simplest complete example
that demonstrates this then. Something like the source for a single
Servlet and a curl command to send a request to it.
Mark
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat 9.0 async read becomes blocking with chunked transfer-encoding
Hi Mark, I’m uploading some test files
GetPayloadServlet.java
Description: Binary data
HttpPostTest.java
Description: Binary data
web.xml
Description: XML document
Below are the steps: 1. compile GetPayloadServlet.java and put it to webapps/test/WEB-INF/classes/test/ 2. put web.xml to webapps/test/WEB-INF/ 3. start tomcat (9.0.19), I tried with 9.0.53, I got the same result 4. use HttpPostTest(I’m using Apache httpclient-4.5.1.jar, httpcore-4.4.4.jar), the test is sending data with chunked transfer-encoding, one step here 4.1. put a debug point before streamWrite at flushBuffer method in org.apache.http.impl.io.SessionOutputBufferImpl, the purpose is to pause flushing to the socket private void flushBuffer() throws IOException { final int len = this.buffer.length(); if (len > 0) { streamWrite(this.buffer.buffer(), 0, len); this.buffer.clear(); this.metrics.incrementBytesTransferred(len); } } 5. put a debug point on GetPayloadServlet @Override public void onDataAvailable() throws IOException {byte buffer[] = new byte[BUFFER_SIZE];while(inputStream.isReady()) {// when isReady returns true, inputStream.read(buffer) is actually blocked with chunked encoding int read = inputStream.read(buffer); if (read < 0) { break; } bos.write(buffer, 0, read);} } 6. we can see once SessionOutputBufferImpl.flushBuffer flushes the data, inputStream.read(buffer) inside GetPayloadServlet will be un-blocked with read Or am I doing something wrong? this is a basic use case. Thanks, AndrewOn Sep 22, 2021, at 1:14 AM, Mark Thomas wrote:On 22/09/2021 08:22, Goldengate liu wrote:Hi Chris,Servlet 3.1 spec defines that ServletInputStream can be used to read as non-blocking way as long as there is data ready locally by calling isReady method and check the ready condition before calling read, and read should throw IllegalStateException if called by caller when data is not readyThe above only applies if the servlet is in async mode. Is it?correct, when it’s running as async modeOK. You are going to need to provide the simplest complete example that demonstrates this then. Something like the source for a single Servlet and a curl command to send a request to it.Mark-To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.orgFor additional commands, e-mail: users-h...@tomcat.apache.org
