Hi Tomcat Team and Mark,
I am able to reproduce the problem with your sample app. *CSP large
header is causing this problem we attached it to ALL responses. This
custom header. I am not sure if this is a tomcat issue or Firefox. *We
get *NS_ERROR_ABORT*. Sorry, it took a while but the issue is
reproducible with CSP or Large header.
Sample:
https://github.com/bmistry13/tomat-204-issue/blob/main/no-content-test.war <https://github.com/bmistry13/tomat-204-issue/blob/main/no-content-test.war>
web.xml
<web-app
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance
<http://www.w3.org/2001/XMLSchema-instance>"
xmlns="http://java.sun.com/xml/ns/javaee
<http://java.sun.com/xml/ns/javaee>"
xmlns:web="http://java.sun.com/xml/ns/javaee/web-app_4_0.xsd
<http://java.sun.com/xml/ns/javaee/web-app_4_0.xsd>"
xsi:schemaLocation="http://java.sun.com/xml/ns/javaee
<http://java.sun.com/xml/ns/javaee>
https://java.sun.com/xml/ns/javaee/web-app_4_0.xsd
<https://java.sun.com/xml/ns/javaee/web-app_4_0.xsd>"
id="Versa"
version="4.0">
<display-name>testa</display-name>
* <servlet>
<description>TESTServlet</description>
<servlet-name>AjaxTestServlet</servlet-name>
<servlet-class>AjaxServletTest</servlet-class>
</servlet>
<servlet-mapping>
<servlet-name>AjaxTestServlet</servlet-name>
<url-pattern>/test</url-pattern>
</servlet-mapping>*
<session-config>
<session-timeout>60</session-timeout>
<tracking-mode>COOKIE</tracking-mode>
<cookie-config>
<http-only>true</http-only>
<secure>true</secure>
</cookie-config>
</session-config>
<security-constraint>
<web-resource-collection>
<web-resource-name>HTTPSOnly</web-resource-name>
<url-pattern>/*</url-pattern>
</web-resource-collection>
</security-constraint>
<security-constraint>
<auth-constraint>
<!-- empty constraint: forbid all access -->
</auth-constraint>
</security-constraint>
<mime-mapping>
<extension>js</extension>
<mime-type>application/javascript</mime-type>
</mime-mapping>
</web-app>
~
--- JSP change
<html>
<body>
<script type="text/javascript"
src="http://ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js
<http://ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js>"></script>
<script type="text/javascript">
function test() {
for (let i = 5; i < 10; i++) {
$.ajax({
async: true,
type: "GET",
url: "/no-content-test/test?q=test1",
contentType: "application/json; charset=utf-8",
dataType: "json",
success: function (response) {
alert(response.status);
}
});
}
for (let i = 5; i < 10; i++) {
$.ajax({
async: true,
type: "GET",
url: "/no-content-test/test?q=test2",
contentType: "application/json; charset=utf-8",
dataType: "json",
success: function (response) {
alert(response.d);
}
});
}
}
</script>
<p>Clinking on button to ajax test <input type='button'
onclick='test();' value='Click Me and See Dev Tool Netowrk Tab'/></p>
</body>
</html>
------
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
public class AjaxServletTest extends HttpServlet {
private static final String CSP_HEADER ="Content-Security-Policy";
private static final String CSP_HEADER_POLICY ="default-src 'self' https:;font-src 'self' data:
https://fonts.googleapis.com/ <https://fonts.googleapis.com/>
https://fonts.gstatic.com <https://fonts.gstatic.com>; script-src 'self' " +
"'sha256-BdZ62JYXk9kaPAizrIertjvay2mtb//aXKRFuOz6RLI='
'sha256-Rv1DFftLIYbmrwNcvKH3vd+fs4aXTb6/RNjTs7CLJOg=' " +
"'sha256-BDvo3wEMDWiXAoVw9mYgGW9GCvsvo3ECBHjc4FzPky8=' 'unsafe-eval'
https://maps.googleapis.com/maps/api/geocode/json
<https://maps.googleapis.com/maps/api/geocode/json>
https://maps.googleapis.com/ <https://maps.googleapis.com/> " +
"https://*.tile.openstreetmap.org <http://tile.openstreetmap.org>
https://export.highcharts.com/ <https://export.highcharts.com/>
https://code.highcharts.com/ <https://code.highcharts.com/>; img-src
'self' blob: " +
"data: https://maps.googleapis.com/maps/api/geocode/json
<https://maps.googleapis.com/maps/api/geocode/json>
https://maps.gstatic.com/ <https://maps.gstatic.com/>
https://maps.googleapis.com/ <https://maps.googleapis.com/>
https://*.tile.openstreetmap.org <http://tile.openstreetmap.org> " +
"https://chart.apis.google.com/ <https://chart.apis.google.com/>
https://export.highcharts.com/ <https://export.highcharts.com/>;
style-src 'self' 'unsafe-inline' data: " +
"https://maps.googleapis.com/maps/api/geocode/json
<https://maps.googleapis.com/maps/api/geocode/json>
https://*.tile.openstreetmap.org <http://tile.openstreetmap.org> " +
"https://fonts.googleapis.com/ <https://fonts.googleapis.com/>
https://export.highcharts.com/ <https://export.highcharts.com/> " +
"https://code.highcharts.com/;form-action
<https://code.highcharts.com/;form-action> 'self'; https:;
block-all-mixed-content; connect-src 'self' wss://self:6080 data:
https://maps.googleapis.com <https://maps.googleapis.com>
https://*.tile.openstreetmap.org/ <http://tile.openstreetmap.org/>
https://ipapi.co <https://ipapi.co> https://oauth2.googleapis.com/token
<https://oauth2.googleapis.com/token> https://dialogflow.googleapis.com
<https://dialogflow.googleapis.com>";
public void doGet(HttpServletRequest req,HttpServletResponse resp)throws
ServletException,IOException {
resp.setStatus(204);
resp.setHeader(CSP_HEADER,CSP_HEADER_POLICY);
}
}
On Wed, Mar 15, 2023 at 11:30 AM Bhavesh Mistry
<mistry.p.bhav...@gmail.com <mailto:mistry.p.bhav...@gmail.com>> wrote:
Hi Mark and Tomcat Team,
We have been using tomcat 9 from version 0 to 70 and no issues so
far with our application and firefox. We also tried to upgrade to
9.0.73, and show the same issue:
As you can see from Devtools it is missing Protocol HTTP2 and
is hung there.
image.png
[04/Mar/2023:00:40:47 +0000] 10.40.207.127 -
https-jsse-nio-127.0.0.1-8443-exec-54 Administrator "GET
/versa/ncs-services/vnms/analyticgroup/all *HTTP/2.0*" 204 - 148ms 148ms
Any clue you have in the filter or why it might be 204 response
caused firefox to be hung? we have added a custom header to the
response that is all. I remove them still same issue. Any theory
or clue you have further to debug this notorious issue?
Thanks,
Bhavesh
On Thu, Mar 9, 2023 at 11:54 AM Mark Thomas <ma...@apache.org
<mailto:ma...@apache.org>> wrote:
On 09/03/2023 18:19, Bhavesh Mistry wrote:
> Hi Mark,
>
> Your sample application worked 204 Firefox and our
application does not
> work. That leads me to believe *Application Filter *is an
issue. But
> should tomcat throw an exception if the response is already
committed?
A call to setStatus() after the response has been committed will
be a NO-OP.
> I
> will try to see if I can reproduce it using a filter with the
sample app
> you gave me. Only one line change
(streamOutputBuffer.closed = true)
> would make our application work. So it seems to me that the
application
> filter is writing something after the stream is closed and
this may lead to
> this behavior. I will try to reproduce using this app. Do
still consider
> this application bug or a tomcat platform bug?
Definitely an application bug.
Mark
>
> Thank you so far for your excellent support and quick responses.
>
> Thanks,
>
> Bhavesh
>
>
>
> On Thu, Mar 9, 2023 at 1:14 AM Mark Thomas <ma...@apache.org
<mailto:ma...@apache.org>> wrote:
>
>> On 08/03/2023 21:32, Bhavesh Mistry wrote:
>>> Hi Mark,
>>>
>>> We have a NAT rule that forwards 443 to 8443.
>>
>> OK. That explains the change in port.
>>
>>> Trust me on that, we have a
>>> direct connection. To rule out any networking layer issues,
I did
>>> direct ssh -L 8443:localhost:8443 admin@10.40.207.140
<mailto:admin@10.40.207.140> and created a
>> tunnel
>>> (https://localhost:8443/ <https://localhost:8443/>) to
bypass port 443. Yet, the issue is still
>>> reproducible. So there is *NOTHING* in the middle that
could cause this
>>> issue.
>>
>> There must be. Could be a FireFox plugin, a Filter used by
the web
>> application, a Valve for a third-party authentication
module, etc.
>>
>> Try deploying this war:
>> https://people.apache.org/~markt/dev/no-content-test.war
<https://people.apache.org/~markt/dev/no-content-test.war>
>>
>> It contains 2 JSPs.
>> index.jsp has a link to no-content.jsp
>> no-content.jsp just returns a 204
>>
>> This works as expected, with no errors with the latest 9.0.x
code,
>> 9.0.72, Chrome and FireFox.
>>
>> If it works when deployed to your server, that points to
something in
>> the web application. If it doesn't work, that points to
something in the
>> network and/or browser.
>>
>> Mark
>>
>> * Is publicly exposing tomcat enough for debugging *or
do you still
>>> need an independent application? I can have SSH open but
you will have
>> to
>>> give me your private email to email credentials and public IP.
>>>
>>> sudo iptables -t nat -L
>>> Chain PREROUTING (policy ACCEPT)
>>> target prot opt source destination
>>> VNMS all -- anywhere anywhere
>>>
>>> Chain INPUT (policy ACCEPT)
>>> target prot opt source destination
>>>
>>> Chain OUTPUT (policy ACCEPT)
>>> target prot opt source destination
>>>
>>> Chain POSTROUTING (policy ACCEPT)
>>> target prot opt source destination
>>> MASQUERADE tcp -- 172.17.0.2 172.17.0.2
tcp
>> dpt:8000
>>>
>>> Chain DOCKER (0 references)
>>> target prot opt source destination
>>> DNAT tcp -- anywhere anywhere
tcp
>> dpt:8000
>>> to:172.17.0.2:8000 <http://172.17.0.2:8000>
>>>
>>> Chain VNMS (1 references)
>>> target prot opt source destination
>>> DNAT tcp -- anywhere anywhere
tcp
>> dpt:http
>>> to:127.0.0.1:8080 <http://127.0.0.1:8080>
>>> *DNAT tcp -- anywhere anywhere
tcp
>>> dpt:https to:127.0.0.1:8443 <http://127.0.0.1:8443>
<http://127.0.0.1:8443 <http://127.0.0.1:8443>> // this rule
>> Fowards
>>> it to the 8443.*
>>> admin@SDWAN-VOAE1:~$
>>>
>>> On Wed, Mar 8, 2023 at 12:29 PM Mark Thomas
<ma...@apache.org <mailto:ma...@apache.org>> wrote:
>>>
>>>> On 08/03/2023 19:52, Bhavesh Mistry wrote:
>>>>> Hi Mark,
>>>>>
>>>>> It is a *direct connection* with no proxy or no reverse
proxy or no
>> load
>>>>> balancer in the middle. We have a web app (UI) that make
backend call
>>>> and
>>>>> return 204 with no content and send it to the browser.
An interesting
>>>> fact
>>>>> is very thing works on Chrome but not with firefox. We
have tried
>>>>> everything and we looked tomcat code and we see a change
log around
>> this
>>>>> area.
>>>>
>>>> That data you provided previously is not consistent with
that statement.
>>>>
>>>> Tomcat is listening on port 8443.
>>>>
>>>> Firefox is connecting to port 443.
>>>>
>>>> I have no doubt the Tomcat change to 204 handling
triggered the change
>>>> in behavior you are seeing. It appears to have exposed a
HTTP/2 bug
>>>> somewhere in your system.
>>>>
>>>>> It is hard to come up with a sample, but I will try it.
I am just
>> trying
>>>>> to give clue which code or line causing the problem to
narrow down the
>>>>> issue, but it is not helping.
>>>>
>>>> Tomcat isn't the root cause. A simple test here with an
index JSP and a
>>>> JSP that just returns a 204 works as expected with Chrome
and FireFox.
>>>>
>>>> All the indications are that there is an additional
component in the
>>>> system you are testing that can't handle an HTTP/2 204
response without
>>>> a body.
>>>>
>>>> Mark
>>>>
>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Bhavesh
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Mar 8, 2023 at 11:43 AM Mark Thomas
<ma...@apache.org <mailto:ma...@apache.org>> wrote:
>>>>>
>>>>>> On 08/03/2023 19:38, Bhavesh Mistry wrote:
>>>>>>> I will see if I can give a sample. But after removing
JUST ONE
>> LINE (
>>>>>>> streamOutputBuffer.closed = true;) Everything seems to
work. Somehow,
>>>>>>> firefox does not like an active stream being closed (I
am not 100%
>> what
>>>>>>> close does).
>>>>>>>
>>>>>>> I will try to work on a sample to reproduce this. I
hope the above
>> can
>>>>>>> give you some clue as to where the issue is.
>>>>>>
>>>>>> Wherever the issue is, it isn't with Tomcat and it isn't
with Firefox.
>>>>>> I've tested them locally and they work correctly.
>>>>>>
>>>>>> Might you have a reverse proxy between Firefox and Tomcat?
>>>>>>
>>>>>> Mark
>>>>>>
>>>>>>
---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail:
users-unsubscr...@tomcat.apache.org
<mailto:users-unsubscr...@tomcat.apache.org>
>>>>>> For additional commands, e-mail:
users-h...@tomcat.apache.org <mailto:users-h...@tomcat.apache.org>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
---------------------------------------------------------------------
>>>> To unsubscribe, e-mail:
users-unsubscr...@tomcat.apache.org
<mailto:users-unsubscr...@tomcat.apache.org>
>>>> For additional commands, e-mail:
users-h...@tomcat.apache.org <mailto:users-h...@tomcat.apache.org>
>>>>
>>>>
>>>
>>
>>
---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
<mailto:users-unsubscr...@tomcat.apache.org>
>> For additional commands, e-mail:
users-h...@tomcat.apache.org <mailto:users-h...@tomcat.apache.org>
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
<mailto:users-unsubscr...@tomcat.apache.org>
For additional commands, e-mail: users-h...@tomcat.apache.org
<mailto:users-h...@tomcat.apache.org>
