On 16/12/2018 03:01, Kohei Nozaki wrote: > Hello, > > It turned out that the "sjsxp" library which JAX-WS RI v2.1.3 uses makes > Tomcat behave this way. I tried a different version of JAX-WS RI (v2.1.7) > which doesn't use the "sjsxp" library anymore and it solved the issue.
Thanks for reporting back when you found a solution. > A very similar issue posted on the Metro mailing list: > http://metro.1045641.n5.nabble.com/JAX-WS-RI-2-1-5-returning-malformed-response-tp1063518.html > > It's surprising that a bug of a framework which is built on the Servlet API > can make such an issue happen, but anyway thank you very much everyone who > helped me out. I'm guessing it held onto a reference and re-used it when it shouldn't. That can cause all sorts of chaos. Mark > > Regards, > Kohei > >> On Dec 4, 2018, at 3:39, Christopher Schultz <ch...@christopherschultz.net> >> wrote: >> > Kohei, > > On 12/1/18 10:06, Kohei Nozaki wrote: >>>> Hello Konstantin, thanks for sharing the valuable information. >>>> >>>>> On Dec 1, 2018, at 19:00, Konstantin Kolinko >>>>> <knst.koli...@gmail.com> wrote: >>>>> >>>>>> * Our downstream Nginx instance (The client of our Tomcat >>>>>> instance) recorded the error "upstream sent no valid HTTP/1.0 >>>>>> header while reading response header from upstream" at that >>>>>> time and the error makes perfect sense concerning the response >>>>>> which has neither HTTP status line nor HTTP headers. >>>>> >>>>> 1. See the official FAQ / Troubleshoting page: >>>>> https://wiki.apache.org/tomcat/FAQ/Troubleshooting_and_Diagnostics >>>>> >>>>> >>>>> > Especially pay attention to >>>>> 1) configuring an access log 2) setting system property >>>>> org.apache.catalina.connector.RECYCLE_FACADES=true >>>> >>>> I've investigated "RECYCLE_FACADES" and understand that an >>>> improperly implemented application which keeps a reference to >>>> Request or Response objects outside of their lifecycle can make >>>> such an issue like mine happen (please correct me if I'm wrong..). >>>> >>>> >>>> But I still don't quite understand what does "RECYCLE_FACADES=true" >>>> do. The Wiki page says "This makes it easier to spot illegal access >>>> when it happens, instead of waiting until side effects of such >>>> access become visible" but how does it make easier? Does this >>>> property make Tomcat produce an Exception or make Tomcat produce >>>> some warning message to a log file or something when such accesses >>>> happen, for example? > > Tomcat usually handles requests something like this. Imagine a > single-threaded server where Tomcat only accepts a single connection > at a time (just to simplify the code to the point where it fits into a > ML post). > > Many of these methods are made-up. There is no > TomcatHttpServletRequest class or a .setRequestLine method in it > (though there are ... siilar concepts in there, way down deep). The > point is how the objects are used, or rather *re* used. > > HttpServletRequest request = new TomcatHttpServletRequest(); > HttpServletResponse response = new TomcatHttpServletResponse(); > > Connection conn = null; > > while(null != (conn = socket.acceptConnection()) { > request.setRequestLine(conn.getRequestLine()); > request.setInputStream(conn.getInputStream()); > response.setOutputStream(conn.getOutputStream()); > > Servlet servlet = getServletForRequest(request); > if(null == servlet) > servlet = defaultServlet; > > servlet.service(request, response); > > request.reset(); > response.reset(); > } > > In "real" Tomcat, each Connection object holds its own Request and > Response objects ad manages them in a similar way, and of course, > Tomcat can accept multiple simultaneous connections -- including > multiple requests over a single connection -- simultaneously -- in the > case of HTTP/2. > > If you enable the RECYCLE_FACADES in Tomcat, the code changes to > behave like this: > > Connection conn = null; > > while(null != (conn = socket.acceptConnection()) { > HttpServletRequest request = new TomcatHttpServletRequest(); > HttpServletResponse response = new TomcatHttpServletResponse(); > request.setRequestLine(conn.getRequestLine()); > request.setInputStream(conn.getInputStream()); > response.setOutputStream(conn.getOutputStream()); > > Servlet servlet = getServletForRequest(request); > if(null == servlet) > servlet = defaultServlet; > > servlet.service(request, response); > > request.dispose(); > response.dispose(); > } > > Note how the request and response objects are no longer re-used across > requests. This represents a trade-off between security/stability > (always getting a fresh object) versus performance (less > garbage-collection for a given request). An application can do things > to the request or response that can break the way the server works, so > an untrusted application should always be run with RECYCLE_FACADES set > to ON. > > If the application keeps a reference to a request or response object > after the request has completed (as defined by returning from > servlet.service()), then Bad Things can happen. If you write to that > response's OutputStream, for example, you might write into the > response of *another request* that is being handled after your code > should be finished. > >>>> >>>> p.s. Speaking of the possibility of an improper implementation, >>>> I've found some discussions which seem to have some similarity to >>>> my case like the following: >>>> >>>> * >>>> http://tomcat.10.x6.nabble.com/NullPointerException-in-MimeHeaders-td2 > 054107.html >>>> : I've seen some similar Exceptions at the "MimeHeaders" class >>>> which were recorded in my Tomcat's log too >>>> >>>> * >>>> http://tomcat.10.x6.nabble.com/Tomcat-occasionally-duplicating-respons > es-td5034710.html >>>> : A case where a bug in a Filter make Tomcat produce an invalid >>>> HTTP response >>>> >>>> Those discussions made me think of this possibility where my app >>>> might be improperly implemented. I'm going to check the application >>>> code from this perspective on Monday. > > When you use RECYCLE_FACADES, your application will only retain a > reference to a useless object.... one which will likely NPE at some > point when your code calls it. This makes the application safer > (because one misbehaving servlet won't damage requests/responses > currently in-use by subsequent requests) and also helps you find the > error in your own code (because the problem will usually manifest > itself as an earlier NPE instead of some "weirdness" which cannot > really be readily explained because of the complexities of how objects > are re-used, etc.). > > Hope that helps, > -chris >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >> <mailto:users-unsubscr...@tomcat.apache.org> >> For additional commands, e-mail: users-h...@tomcat.apache.org >> <mailto:users-h...@tomcat.apache.org> > --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
